2015-02-12 15:45:08,020 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-12 15:45:08,051 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-12 15:45:08,051 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.SortedValuesPlatformTest
2015-02-12 15:45:08,069 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-12 15:45:08,495 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-12 15:45:08,671 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:08,672 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:08,681 INFO property.AppProps (AppProps.java:getAppID(169)) - using app.id: AD6608AB33E946D69A6A1C30E1BD6B7D
2015-02-12 15:45:08,756 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:08,792 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:08,796 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:08,903 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-12 15:45:08,905 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting
2015-02-12 15:45:08,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 15:45:08,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt"]
2015-02-12 15:45:08,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed"]
2015-02-12 15:45:08,907 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] parallel execution is enabled: false
2015-02-12 15:45:08,908 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting jobs: 1
2015-02-12 15:45:08,908 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] allocating threads: 1
2015-02-12 15:45:08,909 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] starting step: (1/1) ...rm/cogroupcomparereversed
2015-02-12 15:45:08,924 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-12 15:45:08,925 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-12 15:45:08,936 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:09,115 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:09,124 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:09,152 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:2
2015-02-12 15:45:09,295 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local2049850255_0001
2015-02-12 15:45:09,320 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2049850255/.staging/job_local2049850255_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:09,327 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2049850255/.staging/job_local2049850255_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:09,450 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2049850255_0001/job_local2049850255_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:09,457 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2049850255_0001/job_local2049850255_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:09,464 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:09,465 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:09,466 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:09,467 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] submitted hadoop job: job_local2049850255_0001
2015-02-12 15:45:09,468 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] tracking url: http://localhost:8080/
2015-02-12 15:45:09,482 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:09,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2049850255_0001_m_000000_0
2015-02-12 15:45:09,512 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:09,513 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:09,521 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt
2015-02-12 15:45:09,521 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@17eb7542
2015-02-12 15:45:09,528 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:09,591 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:09,591 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:09,592 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:09,592 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:09,592 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:09,606 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:09,608 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:09,608 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:09,645 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-12 15:45:09,657 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 15:45:09,658 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:09,711 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:09,712 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:09,712 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:09,712 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 9461; bufvoid = 104857600
2015-02-12 15:45:09,712 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2015-02-12 15:45:15,516 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-12 15:45:18,523 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-12 15:45:21,526 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-12 15:45:23,111 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:23,115 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2049850255_0001_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:23,117 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599
2015-02-12 15:45:23,117 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2049850255_0001_m_000000_0' done.
2015-02-12 15:45:23,117 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2049850255_0001_m_000000_0
2015-02-12 15:45:23,117 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2049850255_0001_m_000001_0
2015-02-12 15:45:23,120 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:23,120 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:23,121 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt
2015-02-12 15:45:23,121 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6b5aba3a
2015-02-12 15:45:23,123 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:23,135 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:23,135 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:23,135 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:23,135 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:23,135 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:23,136 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:23,137 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:23,137 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:23,147 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt"]
2015-02-12 15:45:23,147 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:23,152 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:23,152 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:23,152 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:23,152 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 142; bufvoid = 104857600
2015-02-12 15:45:23,152 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2015-02-12 15:45:24,280 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:24,281 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2049850255_0001_m_000001_0 is done. And is in the process of committing
2015-02-12 15:45:24,283 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt:0+275
2015-02-12 15:45:24,283 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2049850255_0001_m_000001_0' done.
2015-02-12 15:45:24,283 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2049850255_0001_m_000001_0
2015-02-12 15:45:24,283 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:24,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:24,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2049850255_0001_r_000000_0
2015-02-12 15:45:24,290 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:24,290 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:24,291 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@566acddc
2015-02-12 15:45:24,299 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:24,300 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2049850255_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:24,314 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#1 about to shuffle output of map attempt_local2049850255_0001_m_000001_0 decomp: 184 len: 188 to MEMORY
2015-02-12 15:45:24,318 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 184 bytes from map-output for attempt_local2049850255_0001_m_000001_0
2015-02-12 15:45:24,340 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 184, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->184
2015-02-12 15:45:24,343 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#1 about to shuffle output of map attempt_local2049850255_0001_m_000000_0 decomp: 9863 len: 9867 to MEMORY
2015-02-12 15:45:24,344 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 9863 bytes from map-output for attempt_local2049850255_0001_m_000000_0
2015-02-12 15:45:24,344 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 9863, inMemoryMapOutputs.size() -> 2, commitMemory -> 184, usedMemory ->10047
2015-02-12 15:45:24,345 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:24,346 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:24,346 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:24,351 INFO mapred.Merger (Merger.java:merge(606)) - Merging 2 sorted segments
2015-02-12 15:45:24,367 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 2 segments left of total size: 10033 bytes
2015-02-12 15:45:27,003 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 2 segments, 10047 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:27,003 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 10049 bytes from disk
2015-02-12 15:45:27,004 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:27,004 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:27,004 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 10038 bytes
2015-02-12 15:45:27,005 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:27,006 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:27,006 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:27,017 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:27,017 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed"]
2015-02-12 15:45:27,029 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 15:45:27,029 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 15:45:27,054 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2049850255_0001_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:27,055 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:27,055 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local2049850255_0001_r_000000_0 is allowed to commit now
2015-02-12 15:45:27,056 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2049850255_0001_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed/_temporary/0/task_local2049850255_0001_r_000000
2015-02-12 15:45:27,057 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:27,057 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2049850255_0001_r_000000_0' done.
2015-02-12 15:45:27,057 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2049850255_0001_r_000000_0
2015-02-12 15:45:27,057 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:27,085 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed/_temporary
2015-02-12 15:45:27,114 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:27,135 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed
2015-02-12 15:45:27,154 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:27,154 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:27,171 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:27,176 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:27,176 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:27,189 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting
2015-02-12 15:45:27,189 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 15:45:27,189 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt"]
2015-02-12 15:45:27,189 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward"]
2015-02-12 15:45:27,190 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] parallel execution is enabled: false
2015-02-12 15:45:27,190 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting jobs: 1
2015-02-12 15:45:27,190 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] allocating threads: 1
2015-02-12 15:45:27,190 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] starting step: (1/1) ...orm/cogroupcompareforward
2015-02-12 15:45:27,191 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:27,192 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:27,218 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:27,219 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:27,238 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:2
2015-02-12 15:45:27,258 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1341466967_0002
2015-02-12 15:45:27,268 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1341466967/.staging/job_local1341466967_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:27,270 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1341466967/.staging/job_local1341466967_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:27,312 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1341466967_0002/job_local1341466967_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:27,314 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1341466967_0002/job_local1341466967_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:27,319 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:27,320 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:27,321 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:27,321 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] submitted hadoop job: job_local1341466967_0002
2015-02-12 15:45:27,322 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] tracking url: http://localhost:8080/
2015-02-12 15:45:27,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:27,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1341466967_0002_m_000000_0
2015-02-12 15:45:27,324 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:27,325 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:27,325 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt
2015-02-12 15:45:27,326 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@74103471
2015-02-12 15:45:27,327 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:27,343 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:27,344 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:27,344 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:27,344 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:27,344 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:27,345 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:27,345 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:27,346 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:27,354 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 15:45:27,355 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:27,385 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:27,385 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:27,385 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:27,386 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 9461; bufvoid = 104857600
2015-02-12 15:45:27,386 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2015-02-12 15:45:33,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-12 15:45:36,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-12 15:45:38,267 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:38,269 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1341466967_0002_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:38,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599
2015-02-12 15:45:38,270 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1341466967_0002_m_000000_0' done.
2015-02-12 15:45:38,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1341466967_0002_m_000000_0
2015-02-12 15:45:38,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1341466967_0002_m_000001_0
2015-02-12 15:45:38,272 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:38,272 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:38,273 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt
2015-02-12 15:45:38,273 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@51c79d63
2015-02-12 15:45:38,275 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:38,286 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:38,286 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:38,286 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:38,286 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:38,286 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:38,287 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:38,288 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:38,288 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:38,295 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt"]
2015-02-12 15:45:38,295 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:38,298 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:38,298 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:38,298 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:38,298 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 142; bufvoid = 104857600
2015-02-12 15:45:38,298 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2015-02-12 15:45:39,414 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:39,415 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1341466967_0002_m_000001_0 is done. And is in the process of committing
2015-02-12 15:45:39,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/ips.20.txt:0+275
2015-02-12 15:45:39,417 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1341466967_0002_m_000001_0' done.
2015-02-12 15:45:39,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1341466967_0002_m_000001_0
2015-02-12 15:45:39,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:39,418 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:39,418 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1341466967_0002_r_000000_0
2015-02-12 15:45:39,419 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:39,419 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:39,419 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1def1f1e
2015-02-12 15:45:39,420 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:39,421 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1341466967_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:39,422 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#2 about to shuffle output of map attempt_local1341466967_0002_m_000001_0 decomp: 184 len: 188 to MEMORY
2015-02-12 15:45:39,423 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 184 bytes from map-output for attempt_local1341466967_0002_m_000001_0
2015-02-12 15:45:39,423 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 184, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->184
2015-02-12 15:45:39,425 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#2 about to shuffle output of map attempt_local1341466967_0002_m_000000_0 decomp: 9863 len: 9867 to MEMORY
2015-02-12 15:45:39,425 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 9863 bytes from map-output for attempt_local1341466967_0002_m_000000_0
2015-02-12 15:45:39,425 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 9863, inMemoryMapOutputs.size() -> 2, commitMemory -> 184, usedMemory ->10047
2015-02-12 15:45:39,426 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:39,426 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:39,426 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:39,428 INFO mapred.Merger (Merger.java:merge(606)) - Merging 2 sorted segments
2015-02-12 15:45:39,442 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 2 segments left of total size: 10035 bytes
2015-02-12 15:45:41,943 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 2 segments, 10047 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:41,944 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 10049 bytes from disk
2015-02-12 15:45:41,944 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:41,944 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:41,945 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 10039 bytes
2015-02-12 15:45:41,945 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:41,946 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:41,946 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:41,953 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-12 15:45:41,953 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward"]
2015-02-12 15:45:41,961 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 15:45:41,961 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-12 15:45:41,982 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1341466967_0002_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:41,983 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-12 15:45:41,983 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1341466967_0002_r_000000_0 is allowed to commit now
2015-02-12 15:45:41,984 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1341466967_0002_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward/_temporary/0/task_local1341466967_0002_r_000000
2015-02-12 15:45:41,985 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:41,985 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1341466967_0002_r_000000_0' done.
2015-02-12 15:45:41,985 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1341466967_0002_r_000000_0
2015-02-12 15:45:41,985 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:42,009 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward/_temporary
2015-02-12 15:45:42,030 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:42,047 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward
2015-02-12 15:45:42,062 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:42,063 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:42,084 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:42,085 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:42,092 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:42,096 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:42,097 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:42,107 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:42,107 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:42,107 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false"]
2015-02-12 15:45:42,107 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:42,107 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:42,108 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:42,108 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_null_false_false
2015-02-12 15:45:42,109 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:42,110 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:42,139 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:42,160 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:42,209 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local700234736_0003
2015-02-12 15:45:42,224 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi700234736/.staging/job_local700234736_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:42,225 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi700234736/.staging/job_local700234736_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:42,255 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local700234736_0003/job_local700234736_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:42,256 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local700234736_0003/job_local700234736_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:42,256 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:42,256 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local700234736_0003
2015-02-12 15:45:42,256 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:42,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:42,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:42,260 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:42,260 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local700234736_0003_m_000000_0
2015-02-12 15:45:42,261 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:42,261 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:42,262 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:42,263 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5642bf19
2015-02-12 15:45:42,264 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:42,279 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:42,280 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:42,280 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:42,280 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:42,280 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:42,282 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:42,282 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:42,282 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:42,287 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:42,287 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:42,291 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:42,291 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:42,291 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:42,291 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:42,292 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:42,295 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:42,297 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local700234736_0003_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:42,298 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:42,298 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local700234736_0003_m_000000_0' done.
2015-02-12 15:45:42,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local700234736_0003_m_000000_0
2015-02-12 15:45:42,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:42,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:42,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local700234736_0003_r_000000_0
2015-02-12 15:45:42,302 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:42,302 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:42,303 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@11345fac
2015-02-12 15:45:42,304 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:42,305 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local700234736_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:42,307 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#3 about to shuffle output of map attempt_local700234736_0003_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:42,307 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local700234736_0003_m_000000_0
2015-02-12 15:45:42,307 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:42,308 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:42,308 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:42,309 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:42,310 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:42,310 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:42,312 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:42,312 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:42,313 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:42,313 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:42,313 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:42,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:42,314 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:42,314 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:42,319 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:42,319 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false"]
2015-02-12 15:45:42,331 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local700234736_0003_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:42,332 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:42,332 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local700234736_0003_r_000000_0 is allowed to commit now
2015-02-12 15:45:42,333 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local700234736_0003_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false/_temporary/0/task_local700234736_0003_r_000000
2015-02-12 15:45:42,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:42,334 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local700234736_0003_r_000000_0' done.
2015-02-12 15:45:42,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local700234736_0003_r_000000_0
2015-02-12 15:45:42,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:42,353 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false/_temporary
2015-02-12 15:45:42,375 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:42,387 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:42,387 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:42,395 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:42,399 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:42,399 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:42,408 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:42,408 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:42,408 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false"]
2015-02-12 15:45:42,409 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:42,409 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:42,409 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:42,410 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_null_false_false
2015-02-12 15:45:42,410 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:42,411 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:42,440 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:42,466 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:42,482 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1381159654_0004
2015-02-12 15:45:42,491 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1381159654/.staging/job_local1381159654_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:42,492 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1381159654/.staging/job_local1381159654_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:42,525 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1381159654_0004/job_local1381159654_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:42,527 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1381159654_0004/job_local1381159654_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:42,528 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:42,528 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:42,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:42,529 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1381159654_0004
2015-02-12 15:45:42,529 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:42,532 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:42,532 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1381159654_0004_m_000000_0
2015-02-12 15:45:42,533 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:42,533 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:42,534 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:42,534 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@67a177f9
2015-02-12 15:45:42,535 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:42,551 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:42,552 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:42,552 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:42,552 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:42,552 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:42,553 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:42,553 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:42,553 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:42,558 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:42,558 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:42,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:42,563 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:42,563 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:42,563 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:42,564 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:43,816 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:43,818 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1381159654_0004_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:43,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:43,819 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1381159654_0004_m_000000_0' done.
2015-02-12 15:45:43,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1381159654_0004_m_000000_0
2015-02-12 15:45:43,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:43,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:43,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1381159654_0004_r_000000_0
2015-02-12 15:45:43,822 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:43,822 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:43,822 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6f7666de
2015-02-12 15:45:43,823 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:43,824 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1381159654_0004_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:43,825 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#4 about to shuffle output of map attempt_local1381159654_0004_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:43,826 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1381159654_0004_m_000000_0
2015-02-12 15:45:43,826 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:43,827 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:43,828 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:43,828 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:43,830 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:43,830 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:43,832 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:43,832 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:43,833 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:43,833 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:43,833 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:43,834 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:43,834 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:43,834 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:43,839 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:43,839 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false"]
2015-02-12 15:45:43,849 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1381159654_0004_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:43,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:43,851 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1381159654_0004_r_000000_0 is allowed to commit now
2015-02-12 15:45:43,852 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1381159654_0004_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false/_temporary/0/task_local1381159654_0004_r_000000
2015-02-12 15:45:43,853 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:43,853 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1381159654_0004_r_000000_0' done.
2015-02-12 15:45:43,853 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1381159654_0004_r_000000_0
2015-02-12 15:45:43,853 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:43,879 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false/_temporary
2015-02-12 15:45:43,901 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:43,914 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:43,915 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:43,921 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:43,923 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:43,924 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:43,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:43,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:43,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false"]
2015-02-12 15:45:43,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:43,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:43,935 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:43,936 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_true_false
2015-02-12 15:45:43,936 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:43,938 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:43,959 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:43,978 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:44,002 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1609188861_0005
2015-02-12 15:45:44,011 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1609188861/.staging/job_local1609188861_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:44,012 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1609188861/.staging/job_local1609188861_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:44,039 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1609188861_0005/job_local1609188861_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:44,040 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1609188861_0005/job_local1609188861_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:44,040 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:44,040 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:44,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:44,040 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1609188861_0005
2015-02-12 15:45:44,041 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:44,043 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:44,043 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1609188861_0005_m_000000_0
2015-02-12 15:45:44,045 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:44,045 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:44,046 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:44,046 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5eef3d7b
2015-02-12 15:45:44,047 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:44,063 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:44,064 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:44,064 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:44,064 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:44,064 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:44,065 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:44,065 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:44,066 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:44,072 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:44,072 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:44,076 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:44,076 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:44,076 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:44,077 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:44,077 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:44,079 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:44,081 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1609188861_0005_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:44,083 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:44,083 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1609188861_0005_m_000000_0' done.
2015-02-12 15:45:44,083 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1609188861_0005_m_000000_0
2015-02-12 15:45:44,084 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:44,084 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:44,084 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1609188861_0005_r_000000_0
2015-02-12 15:45:44,085 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:44,086 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:44,086 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42e18b85
2015-02-12 15:45:44,086 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:44,087 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1609188861_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:44,088 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#5 about to shuffle output of map attempt_local1609188861_0005_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:44,088 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1609188861_0005_m_000000_0
2015-02-12 15:45:44,088 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:44,088 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:44,089 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:44,089 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:44,091 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:44,091 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:44,092 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:44,092 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:44,092 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:44,092 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:44,093 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:44,093 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:44,093 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:44,093 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:44,097 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:44,098 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false"]
2015-02-12 15:45:44,110 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1609188861_0005_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:44,111 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:44,111 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1609188861_0005_r_000000_0 is allowed to commit now
2015-02-12 15:45:44,112 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1609188861_0005_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false/_temporary/0/task_local1609188861_0005_r_000000
2015-02-12 15:45:44,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:44,113 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1609188861_0005_r_000000_0' done.
2015-02-12 15:45:44,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1609188861_0005_r_000000_0
2015-02-12 15:45:44,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:44,131 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false/_temporary
2015-02-12 15:45:44,151 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:44,162 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:44,163 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:44,168 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:44,171 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:44,172 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false"]
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:44,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:44,179 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_true_false
2015-02-12 15:45:44,179 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:44,180 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:44,201 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:44,220 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:44,235 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1295767555_0006
2015-02-12 15:45:44,246 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1295767555/.staging/job_local1295767555_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:44,247 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1295767555/.staging/job_local1295767555_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:44,275 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1295767555_0006/job_local1295767555_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:44,276 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1295767555_0006/job_local1295767555_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:44,277 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:44,277 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1295767555_0006
2015-02-12 15:45:44,277 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:44,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:44,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:44,280 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:44,280 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1295767555_0006_m_000000_0
2015-02-12 15:45:44,282 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:44,282 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:44,282 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:44,283 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2923ac47
2015-02-12 15:45:44,283 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:44,300 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:44,301 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:44,301 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:44,301 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:44,301 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:44,302 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:44,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:44,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:44,307 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:44,307 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:44,310 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:44,310 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:44,310 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:44,311 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:44,311 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:45,553 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:45,554 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1295767555_0006_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:45,555 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:45,555 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1295767555_0006_m_000000_0' done.
2015-02-12 15:45:45,555 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1295767555_0006_m_000000_0
2015-02-12 15:45:45,556 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:45,556 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:45,556 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1295767555_0006_r_000000_0
2015-02-12 15:45:45,558 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:45,558 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:45,558 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@226eaedd
2015-02-12 15:45:45,558 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:45,559 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1295767555_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:45,560 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#6 about to shuffle output of map attempt_local1295767555_0006_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:45,560 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1295767555_0006_m_000000_0
2015-02-12 15:45:45,560 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:45,561 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:45,561 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,561 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:45,563 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:45,563 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:45,565 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:45,565 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:45,565 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:45,565 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:45,566 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:45,566 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,566 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:45,567 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:45,571 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:45,571 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false"]
2015-02-12 15:45:45,581 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1295767555_0006_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:45,582 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,582 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1295767555_0006_r_000000_0 is allowed to commit now
2015-02-12 15:45:45,583 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1295767555_0006_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false/_temporary/0/task_local1295767555_0006_r_000000
2015-02-12 15:45:45,584 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:45,584 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1295767555_0006_r_000000_0' done.
2015-02-12 15:45:45,584 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1295767555_0006_r_000000_0
2015-02-12 15:45:45,584 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:45,604 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false/_temporary
2015-02-12 15:45:45,625 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:45,635 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:45,635 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:45,640 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:45,649 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:45,649 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:45,658 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:45,658 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:45,658 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false"]
2015-02-12 15:45:45,658 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:45,659 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:45,659 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:45,659 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...e/false_false_false_false
2015-02-12 15:45:45,660 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:45,661 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:45,687 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:45,706 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:45,720 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1886098531_0007
2015-02-12 15:45:45,730 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1886098531/.staging/job_local1886098531_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:45,731 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1886098531/.staging/job_local1886098531_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:45,759 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1886098531_0007/job_local1886098531_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:45,759 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1886098531_0007/job_local1886098531_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:45,760 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:45,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:45,760 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1886098531_0007
2015-02-12 15:45:45,760 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:45,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:45,762 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:45,762 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1886098531_0007_m_000000_0
2015-02-12 15:45:45,764 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:45,764 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:45,765 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:45,765 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1f509a03
2015-02-12 15:45:45,766 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:45,780 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:45,780 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:45,780 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:45,780 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:45,781 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:45,782 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:45,782 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:45,782 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:45,788 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:45,789 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:45,793 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:45,793 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:45,793 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:45,793 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:45,794 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:45,796 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:45,799 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1886098531_0007_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:45,800 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:45,800 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1886098531_0007_m_000000_0' done.
2015-02-12 15:45:45,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1886098531_0007_m_000000_0
2015-02-12 15:45:45,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:45,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:45,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1886098531_0007_r_000000_0
2015-02-12 15:45:45,805 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:45,805 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:45,805 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c7c2133
2015-02-12 15:45:45,807 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:45,808 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1886098531_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:45,809 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#7 about to shuffle output of map attempt_local1886098531_0007_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:45,810 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1886098531_0007_m_000000_0
2015-02-12 15:45:45,810 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:45,810 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:45,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,811 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:45,813 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:45,813 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:45,815 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:45,815 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:45,815 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:45,815 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:45,815 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:45,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,816 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:45,816 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:45,822 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:45,822 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false"]
2015-02-12 15:45:45,833 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1886098531_0007_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:45,834 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:45,834 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1886098531_0007_r_000000_0 is allowed to commit now
2015-02-12 15:45:45,835 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1886098531_0007_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false/_temporary/0/task_local1886098531_0007_r_000000
2015-02-12 15:45:45,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:45,836 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1886098531_0007_r_000000_0' done.
2015-02-12 15:45:45,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1886098531_0007_r_000000_0
2015-02-12 15:45:45,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:45,859 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false/_temporary
2015-02-12 15:45:45,880 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:45,890 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:45,891 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:45,895 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:45,898 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:45,899 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:45,905 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:45,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:45,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false"]
2015-02-12 15:45:45,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:45,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:45,906 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:45,906 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/true_false_false_false
2015-02-12 15:45:45,907 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:45,908 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:45,931 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:45,951 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:45,966 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local2067659682_0008
2015-02-12 15:45:45,979 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2067659682/.staging/job_local2067659682_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:45,980 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2067659682/.staging/job_local2067659682_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:46,008 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2067659682_0008/job_local2067659682_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:46,009 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2067659682_0008/job_local2067659682_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:46,010 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:46,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:46,010 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local2067659682_0008
2015-02-12 15:45:46,010 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:46,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:46,012 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:46,012 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2067659682_0008_m_000000_0
2015-02-12 15:45:46,013 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:46,013 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:46,014 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:46,014 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3129517
2015-02-12 15:45:46,015 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:46,032 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:46,032 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:46,032 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:46,032 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:46,033 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:46,034 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:46,034 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:46,034 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:46,038 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:46,038 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:46,042 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:46,042 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:46,042 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:46,042 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:46,042 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:47,883 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:47,884 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2067659682_0008_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:47,885 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:47,885 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2067659682_0008_m_000000_0' done.
2015-02-12 15:45:47,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2067659682_0008_m_000000_0
2015-02-12 15:45:47,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:47,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:47,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2067659682_0008_r_000000_0
2015-02-12 15:45:47,888 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:47,888 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:47,888 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@722dbece
2015-02-12 15:45:47,888 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:47,889 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2067659682_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:47,890 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#8 about to shuffle output of map attempt_local2067659682_0008_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:47,890 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local2067659682_0008_m_000000_0
2015-02-12 15:45:47,890 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:47,891 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:47,891 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:47,891 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:47,893 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:47,893 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:47,894 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:47,895 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:47,895 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:47,895 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:47,895 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:47,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:47,896 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:47,896 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:47,900 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:47,900 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false"]
2015-02-12 15:45:47,910 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2067659682_0008_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:47,911 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:47,911 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local2067659682_0008_r_000000_0 is allowed to commit now
2015-02-12 15:45:47,912 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2067659682_0008_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false/_temporary/0/task_local2067659682_0008_r_000000
2015-02-12 15:45:47,913 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:47,913 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2067659682_0008_r_000000_0' done.
2015-02-12 15:45:47,913 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2067659682_0008_r_000000_0
2015-02-12 15:45:47,913 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:47,924 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false/_temporary
2015-02-12 15:45:47,950 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:47,962 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:47,962 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:47,968 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:47,970 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:47,972 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:47,984 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:47,984 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:47,984 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false"]
2015-02-12 15:45:47,984 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:47,984 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:47,985 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:47,985 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_true_false_false
2015-02-12 15:45:47,985 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:47,987 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:48,016 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:48,041 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:48,061 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1850013092_0009
2015-02-12 15:45:48,081 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1850013092/.staging/job_local1850013092_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:48,082 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1850013092/.staging/job_local1850013092_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:48,121 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1850013092_0009/job_local1850013092_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:48,123 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1850013092_0009/job_local1850013092_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:48,124 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:48,124 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:48,125 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:48,127 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1850013092_0009
2015-02-12 15:45:48,127 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1850013092_0009_m_000000_0
2015-02-12 15:45:48,127 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:48,127 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:48,129 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:48,129 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:48,130 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:48,130 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@673a7418
2015-02-12 15:45:48,131 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:48,152 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:48,152 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:48,152 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:48,152 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:48,153 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:48,154 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:48,154 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:48,154 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:48,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:48,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:48,165 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:48,165 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:48,165 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:48,165 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:48,165 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:50,580 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:50,582 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1850013092_0009_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:50,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:50,583 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1850013092_0009_m_000000_0' done.
2015-02-12 15:45:50,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1850013092_0009_m_000000_0
2015-02-12 15:45:50,583 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:50,584 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:50,584 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1850013092_0009_r_000000_0
2015-02-12 15:45:50,585 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:50,585 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:50,585 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2d6866c6
2015-02-12 15:45:50,587 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:50,587 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1850013092_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:50,588 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#9 about to shuffle output of map attempt_local1850013092_0009_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:50,589 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1850013092_0009_m_000000_0
2015-02-12 15:45:50,589 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:50,589 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:50,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:50,590 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:50,591 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:50,591 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:50,593 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:50,593 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:50,593 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:50,593 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:50,594 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:50,594 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:50,595 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:50,595 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:50,599 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:50,599 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false"]
2015-02-12 15:45:50,609 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1850013092_0009_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:50,610 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:50,610 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1850013092_0009_r_000000_0 is allowed to commit now
2015-02-12 15:45:50,611 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1850013092_0009_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false/_temporary/0/task_local1850013092_0009_r_000000
2015-02-12 15:45:50,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:50,612 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1850013092_0009_r_000000_0' done.
2015-02-12 15:45:50,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1850013092_0009_r_000000_0
2015-02-12 15:45:50,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:50,627 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false/_temporary
2015-02-12 15:45:50,652 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:50,662 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:50,663 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:50,667 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:50,669 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:50,671 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false"]
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:50,678 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:50,679 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_true_false_false
2015-02-12 15:45:50,679 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:50,680 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:50,701 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:50,718 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:50,736 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local271878009_0010
2015-02-12 15:45:50,748 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi271878009/.staging/job_local271878009_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:50,749 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi271878009/.staging/job_local271878009_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:50,778 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local271878009_0010/job_local271878009_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:50,780 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local271878009_0010/job_local271878009_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:50,781 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:50,781 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:50,781 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:50,781 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local271878009_0010
2015-02-12 15:45:50,782 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:50,783 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:50,784 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local271878009_0010_m_000000_0
2015-02-12 15:45:50,785 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:50,785 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:50,786 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:50,786 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ff44435
2015-02-12 15:45:50,787 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:50,807 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:50,807 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:50,807 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:50,807 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:50,807 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:50,808 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:50,808 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:50,808 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:50,812 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:50,813 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:50,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:50,816 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:50,816 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:50,816 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:50,816 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:51,986 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:51,987 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local271878009_0010_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:51,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:51,989 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local271878009_0010_m_000000_0' done.
2015-02-12 15:45:51,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local271878009_0010_m_000000_0
2015-02-12 15:45:51,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:51,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:51,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local271878009_0010_r_000000_0
2015-02-12 15:45:51,991 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:51,991 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:51,991 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@19a81722
2015-02-12 15:45:51,991 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:51,992 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local271878009_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:51,993 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#10 about to shuffle output of map attempt_local271878009_0010_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:51,993 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local271878009_0010_m_000000_0
2015-02-12 15:45:51,994 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:51,994 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:51,994 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:51,995 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:51,996 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:51,996 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:51,998 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:51,998 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:51,998 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:51,998 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:51,999 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:51,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:51,999 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:51,999 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:52,003 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:52,003 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false"]
2015-02-12 15:45:52,012 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local271878009_0010_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:52,013 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:52,013 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local271878009_0010_r_000000_0 is allowed to commit now
2015-02-12 15:45:52,014 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local271878009_0010_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false/_temporary/0/task_local271878009_0010_r_000000
2015-02-12 15:45:52,014 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:52,015 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local271878009_0010_r_000000_0' done.
2015-02-12 15:45:52,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local271878009_0010_r_000000_0
2015-02-12 15:45:52,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:52,027 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false/_temporary
2015-02-12 15:45:52,049 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:52,061 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:52,061 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:52,066 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:52,068 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:52,068 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:52,076 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:52,076 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:52,076 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false"]
2015-02-12 15:45:52,076 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:52,077 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:52,077 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:52,079 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_true_false
2015-02-12 15:45:52,079 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:52,080 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:52,105 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:52,124 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:52,137 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local2093541652_0011
2015-02-12 15:45:52,146 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2093541652/.staging/job_local2093541652_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:52,147 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi2093541652/.staging/job_local2093541652_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:52,183 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2093541652_0011/job_local2093541652_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:52,183 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local2093541652_0011/job_local2093541652_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:52,184 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:52,184 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:52,184 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local2093541652_0011
2015-02-12 15:45:52,184 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:52,184 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:52,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:52,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2093541652_0011_m_000000_0
2015-02-12 15:45:52,187 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:52,187 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:52,188 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:52,188 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@782b1040
2015-02-12 15:45:52,189 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:52,201 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:52,201 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:52,201 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:52,201 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:52,201 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:52,202 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:52,202 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:52,202 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:52,207 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:52,207 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:52,210 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:52,211 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:52,211 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:52,211 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:52,211 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:52,213 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:52,215 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2093541652_0011_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:52,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:52,216 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2093541652_0011_m_000000_0' done.
2015-02-12 15:45:52,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2093541652_0011_m_000000_0
2015-02-12 15:45:52,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:52,217 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:52,217 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2093541652_0011_r_000000_0
2015-02-12 15:45:52,218 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:52,218 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:52,219 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@69e9261d
2015-02-12 15:45:52,219 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:52,221 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2093541652_0011_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:52,222 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#11 about to shuffle output of map attempt_local2093541652_0011_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:52,223 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local2093541652_0011_m_000000_0
2015-02-12 15:45:52,223 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:52,223 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:52,224 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:52,224 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:52,225 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:52,225 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:52,226 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:52,227 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:52,227 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:52,227 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:52,227 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:52,228 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:52,228 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:52,228 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:52,232 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:52,232 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false"]
2015-02-12 15:45:52,246 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local2093541652_0011_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:52,247 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:52,247 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local2093541652_0011_r_000000_0 is allowed to commit now
2015-02-12 15:45:52,248 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2093541652_0011_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false/_temporary/0/task_local2093541652_0011_r_000000
2015-02-12 15:45:52,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:52,249 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2093541652_0011_r_000000_0' done.
2015-02-12 15:45:52,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2093541652_0011_r_000000_0
2015-02-12 15:45:52,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:52,273 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false/_temporary
2015-02-12 15:45:52,294 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:52,308 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:52,308 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:52,313 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:52,314 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:52,314 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false"]
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:52,320 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:52,321 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_true_false
2015-02-12 15:45:52,321 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:52,322 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:52,343 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:52,361 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:52,374 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1137491527_0012
2015-02-12 15:45:52,383 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1137491527/.staging/job_local1137491527_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:52,384 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1137491527/.staging/job_local1137491527_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:52,412 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1137491527_0012/job_local1137491527_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:52,413 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1137491527_0012/job_local1137491527_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:52,413 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:52,414 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1137491527_0012
2015-02-12 15:45:52,414 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:52,414 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:52,414 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:52,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:52,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1137491527_0012_m_000000_0
2015-02-12 15:45:52,417 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:52,417 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:52,418 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:52,418 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@61afff7a
2015-02-12 15:45:52,419 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:52,433 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:52,433 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:52,433 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:52,433 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:52,433 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:52,435 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:52,435 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:52,435 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:52,440 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:52,440 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:52,444 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:52,444 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:52,444 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:52,444 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:52,444 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:54,347 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:54,348 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1137491527_0012_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:54,349 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:54,350 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1137491527_0012_m_000000_0' done.
2015-02-12 15:45:54,350 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1137491527_0012_m_000000_0
2015-02-12 15:45:54,350 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:54,350 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:54,351 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1137491527_0012_r_000000_0
2015-02-12 15:45:54,352 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:54,352 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:54,352 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@289d8a32
2015-02-12 15:45:54,352 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:54,353 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1137491527_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:54,354 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#12 about to shuffle output of map attempt_local1137491527_0012_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:54,354 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1137491527_0012_m_000000_0
2015-02-12 15:45:54,354 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:54,355 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:54,355 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:54,355 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:54,357 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:54,357 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:54,358 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:54,359 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:54,359 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:54,359 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:54,359 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:54,360 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:54,360 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:54,360 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:54,364 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:54,364 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false"]
2015-02-12 15:45:54,376 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1137491527_0012_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:54,377 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:54,377 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1137491527_0012_r_000000_0 is allowed to commit now
2015-02-12 15:45:54,378 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1137491527_0012_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false/_temporary/0/task_local1137491527_0012_r_000000
2015-02-12 15:45:54,378 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:54,379 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1137491527_0012_r_000000_0' done.
2015-02-12 15:45:54,379 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1137491527_0012_r_000000_0
2015-02-12 15:45:54,379 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:54,399 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false/_temporary
2015-02-12 15:45:54,419 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:54,430 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:54,430 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:54,436 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:54,438 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:54,440 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:54,455 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:54,455 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:54,456 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false"]
2015-02-12 15:45:54,456 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:54,456 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:54,456 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:54,456 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_true_false
2015-02-12 15:45:54,457 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:54,458 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:54,491 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:54,516 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:54,537 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local857957513_0013
2015-02-12 15:45:54,553 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi857957513/.staging/job_local857957513_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:54,554 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi857957513/.staging/job_local857957513_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:54,585 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local857957513_0013/job_local857957513_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:54,586 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local857957513_0013/job_local857957513_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:54,586 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:54,586 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:54,587 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:54,587 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local857957513_0013
2015-02-12 15:45:54,587 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:54,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:54,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local857957513_0013_m_000000_0
2015-02-12 15:45:54,590 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:54,590 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:54,591 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:54,591 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12c0d9f9
2015-02-12 15:45:54,591 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:54,607 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:54,607 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:54,607 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:54,607 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:54,608 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:54,609 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:54,609 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:54,609 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:54,615 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:54,615 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:54,619 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:54,619 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:54,619 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:54,620 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:54,620 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:56,955 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:56,956 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local857957513_0013_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:56,957 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:56,957 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local857957513_0013_m_000000_0' done.
2015-02-12 15:45:56,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local857957513_0013_m_000000_0
2015-02-12 15:45:56,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:56,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:56,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local857957513_0013_r_000000_0
2015-02-12 15:45:56,959 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:56,959 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:56,960 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@36f26790
2015-02-12 15:45:56,961 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:56,961 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local857957513_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:56,962 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#13 about to shuffle output of map attempt_local857957513_0013_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:56,963 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local857957513_0013_m_000000_0
2015-02-12 15:45:56,963 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:56,963 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:56,964 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:56,964 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:56,965 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:56,965 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:56,967 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:56,967 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:56,967 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:56,967 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:56,968 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:56,968 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:56,969 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:56,969 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:56,972 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:56,972 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false"]
2015-02-12 15:45:56,981 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local857957513_0013_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:56,982 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:56,982 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local857957513_0013_r_000000_0 is allowed to commit now
2015-02-12 15:45:56,983 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local857957513_0013_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false/_temporary/0/task_local857957513_0013_r_000000
2015-02-12 15:45:56,984 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:56,984 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local857957513_0013_r_000000_0' done.
2015-02-12 15:45:56,984 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local857957513_0013_r_000000_0
2015-02-12 15:45:56,984 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:57,002 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false/_temporary
2015-02-12 15:45:57,021 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:57,031 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:57,031 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:57,035 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:57,036 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:57,036 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:57,042 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:57,043 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:57,043 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false"]
2015-02-12 15:45:57,043 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:57,043 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:57,043 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:57,043 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_true_false
2015-02-12 15:45:57,044 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:57,045 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:57,066 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:57,084 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:57,096 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local276403142_0014
2015-02-12 15:45:57,104 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi276403142/.staging/job_local276403142_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:57,105 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi276403142/.staging/job_local276403142_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:57,143 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local276403142_0014/job_local276403142_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:57,144 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local276403142_0014/job_local276403142_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:57,144 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:57,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:57,144 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local276403142_0014
2015-02-12 15:45:57,145 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:57,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:57,146 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:57,146 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local276403142_0014_m_000000_0
2015-02-12 15:45:57,148 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:57,148 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:57,149 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:57,149 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@353cadff
2015-02-12 15:45:57,150 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:57,166 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:57,166 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:57,166 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:57,166 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:57,167 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:57,168 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:57,168 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:57,168 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:57,172 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:57,172 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:57,177 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:57,177 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:57,177 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:57,177 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:57,177 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:58,396 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:58,397 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local276403142_0014_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:58,398 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local276403142_0014_m_000000_0' done.
2015-02-12 15:45:58,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local276403142_0014_m_000000_0
2015-02-12 15:45:58,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:58,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:58,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local276403142_0014_r_000000_0
2015-02-12 15:45:58,400 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:58,400 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:58,400 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@341349cd
2015-02-12 15:45:58,401 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:58,401 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local276403142_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:58,402 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#14 about to shuffle output of map attempt_local276403142_0014_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:58,402 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local276403142_0014_m_000000_0
2015-02-12 15:45:58,402 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:58,403 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:58,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,403 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:58,405 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,405 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:58,406 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:58,406 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:58,406 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:58,406 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,407 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:58,407 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,407 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:58,408 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:58,412 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:58,412 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false"]
2015-02-12 15:45:58,423 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local276403142_0014_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,424 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,424 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local276403142_0014_r_000000_0 is allowed to commit now
2015-02-12 15:45:58,425 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local276403142_0014_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false/_temporary/0/task_local276403142_0014_r_000000
2015-02-12 15:45:58,426 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:58,426 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local276403142_0014_r_000000_0' done.
2015-02-12 15:45:58,426 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local276403142_0014_r_000000_0
2015-02-12 15:45:58,426 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:58,446 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false/_temporary
2015-02-12 15:45:58,465 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:58,474 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:58,474 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:58,478 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:58,480 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:58,480 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true"]
2015-02-12 15:45:58,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:58,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:58,489 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:58,489 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_false_true
2015-02-12 15:45:58,490 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,491 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,513 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:58,534 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:58,547 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1506142222_0015
2015-02-12 15:45:58,555 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1506142222/.staging/job_local1506142222_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:58,556 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1506142222/.staging/job_local1506142222_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:58,591 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1506142222_0015/job_local1506142222_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:58,592 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1506142222_0015/job_local1506142222_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:58,592 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:58,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:58,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:58,594 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1506142222_0015
2015-02-12 15:45:58,594 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:58,595 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:58,596 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1506142222_0015_m_000000_0
2015-02-12 15:45:58,597 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:58,597 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:58,597 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:58,598 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@38571c65
2015-02-12 15:45:58,598 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:58,612 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:58,612 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:58,612 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:58,612 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:58,612 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:58,614 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:58,614 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:58,614 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:58,619 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:58,619 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:58,623 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:58,624 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:58,624 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:58,624 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:58,624 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:58,627 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:58,628 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1506142222_0015_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,629 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:58,630 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1506142222_0015_m_000000_0' done.
2015-02-12 15:45:58,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1506142222_0015_m_000000_0
2015-02-12 15:45:58,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:58,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:58,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1506142222_0015_r_000000_0
2015-02-12 15:45:58,631 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:58,631 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:58,631 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@20fb7fb8
2015-02-12 15:45:58,632 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:58,632 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1506142222_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:58,633 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#15 about to shuffle output of map attempt_local1506142222_0015_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:58,634 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1506142222_0015_m_000000_0
2015-02-12 15:45:58,634 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:58,634 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:58,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,635 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:58,636 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,637 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:58,638 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:58,638 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:58,638 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:58,639 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,639 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:58,639 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,640 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:58,640 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:58,645 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:58,646 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true"]
2015-02-12 15:45:58,658 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1506142222_0015_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,659 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,660 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1506142222_0015_r_000000_0 is allowed to commit now
2015-02-12 15:45:58,660 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1506142222_0015_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true/_temporary/0/task_local1506142222_0015_r_000000
2015-02-12 15:45:58,661 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:58,661 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1506142222_0015_r_000000_0' done.
2015-02-12 15:45:58,662 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1506142222_0015_r_000000_0
2015-02-12 15:45:58,662 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:58,673 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true/_temporary
2015-02-12 15:45:58,695 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:58,706 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:58,706 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:58,712 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:58,714 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:58,715 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:58,721 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true"]
2015-02-12 15:45:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:58,722 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:58,722 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_false_true
2015-02-12 15:45:58,723 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,746 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:58,763 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:58,775 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1614659200_0016
2015-02-12 15:45:58,785 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1614659200/.staging/job_local1614659200_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:58,786 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1614659200/.staging/job_local1614659200_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:58,815 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1614659200_0016/job_local1614659200_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:58,816 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1614659200_0016/job_local1614659200_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:58,816 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:58,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:58,816 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1614659200_0016
2015-02-12 15:45:58,817 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:58,817 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:58,818 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:58,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1614659200_0016_m_000000_0
2015-02-12 15:45:58,820 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:58,821 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:58,822 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:58,822 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4cd28fca
2015-02-12 15:45:58,824 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:58,842 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:58,842 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:58,842 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:58,842 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:58,843 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:58,844 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:58,844 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:58,844 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:58,849 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:58,849 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:58,852 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:58,852 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:58,852 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:58,852 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:58,853 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:58,854 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:58,856 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1614659200_0016_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,857 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:58,857 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1614659200_0016_m_000000_0' done.
2015-02-12 15:45:58,857 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1614659200_0016_m_000000_0
2015-02-12 15:45:58,857 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:58,857 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:58,858 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1614659200_0016_r_000000_0
2015-02-12 15:45:58,859 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:58,859 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:58,859 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@21f4efa3
2015-02-12 15:45:58,860 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:58,860 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1614659200_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:58,861 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#16 about to shuffle output of map attempt_local1614659200_0016_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:58,862 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1614659200_0016_m_000000_0
2015-02-12 15:45:58,862 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:58,862 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:58,862 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,863 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:58,864 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,864 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:58,866 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:58,866 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:58,866 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:58,866 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:58,866 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:58,867 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,868 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:58,868 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:58,872 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:58,872 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true"]
2015-02-12 15:45:58,884 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1614659200_0016_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:58,885 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:58,885 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1614659200_0016_r_000000_0 is allowed to commit now
2015-02-12 15:45:58,886 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1614659200_0016_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true/_temporary/0/task_local1614659200_0016_r_000000
2015-02-12 15:45:58,886 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:58,887 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1614659200_0016_r_000000_0' done.
2015-02-12 15:45:58,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1614659200_0016_r_000000_0
2015-02-12 15:45:58,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:58,914 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true/_temporary
2015-02-12 15:45:58,934 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:58,946 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:58,947 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:58,951 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:58,952 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:58,952 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:58,957 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:58,958 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:58,958 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true"]
2015-02-12 15:45:58,958 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:58,958 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:58,958 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:58,959 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/false_null_true_true
2015-02-12 15:45:58,959 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,960 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:58,983 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,000 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:59,013 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local337600695_0017
2015-02-12 15:45:59,021 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi337600695/.staging/job_local337600695_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,022 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi337600695/.staging/job_local337600695_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,049 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local337600695_0017/job_local337600695_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,050 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local337600695_0017/job_local337600695_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,051 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:59,051 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local337600695_0017
2015-02-12 15:45:59,051 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:59,051 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:59,051 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:59,053 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:59,053 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local337600695_0017_m_000000_0
2015-02-12 15:45:59,054 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,054 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,055 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:59,055 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@265641cd
2015-02-12 15:45:59,056 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:59,073 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:59,073 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:59,073 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:59,073 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:59,073 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:59,074 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:59,074 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:59,074 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,079 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,079 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,082 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:59,082 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:59,082 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:59,082 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:59,082 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:59,084 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:59,085 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local337600695_0017_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:59,086 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local337600695_0017_m_000000_0' done.
2015-02-12 15:45:59,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local337600695_0017_m_000000_0
2015-02-12 15:45:59,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:59,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:59,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local337600695_0017_r_000000_0
2015-02-12 15:45:59,088 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,088 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,088 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@17570e08
2015-02-12 15:45:59,089 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:59,089 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local337600695_0017_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:59,090 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#17 about to shuffle output of map attempt_local337600695_0017_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:59,090 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local337600695_0017_m_000000_0
2015-02-12 15:45:59,091 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:59,091 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:59,092 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,092 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:59,093 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,094 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:59,095 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:59,095 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:59,096 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:59,096 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,096 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:59,097 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,097 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:59,097 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,101 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,101 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true"]
2015-02-12 15:45:59,110 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local337600695_0017_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,111 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,111 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local337600695_0017_r_000000_0 is allowed to commit now
2015-02-12 15:45:59,112 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local337600695_0017_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true/_temporary/0/task_local337600695_0017_r_000000
2015-02-12 15:45:59,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:59,113 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local337600695_0017_r_000000_0' done.
2015-02-12 15:45:59,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local337600695_0017_r_000000_0
2015-02-12 15:45:59,114 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:59,130 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true/_temporary
2015-02-12 15:45:59,151 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,160 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:59,160 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:59,165 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:59,166 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:59,166 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:59,171 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:59,172 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,172 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true"]
2015-02-12 15:45:59,172 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:59,172 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:59,172 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:59,173 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...nsive/true_null_true_true
2015-02-12 15:45:59,173 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,174 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,195 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,211 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:59,225 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1388807078_0018
2015-02-12 15:45:59,233 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1388807078/.staging/job_local1388807078_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,234 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1388807078/.staging/job_local1388807078_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,260 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1388807078_0018/job_local1388807078_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,261 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1388807078_0018/job_local1388807078_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,262 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:59,262 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:59,262 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:59,262 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1388807078_0018
2015-02-12 15:45:59,262 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:59,265 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:59,265 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1388807078_0018_m_000000_0
2015-02-12 15:45:59,267 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,267 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,267 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:59,268 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@77a13499
2015-02-12 15:45:59,268 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:59,285 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:59,285 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:59,285 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:59,285 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:59,285 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:59,286 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:59,286 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:59,286 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,290 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,290 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,294 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:59,294 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:59,294 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:59,294 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-12 15:45:59,294 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:59,297 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:59,298 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1388807078_0018_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:59,299 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1388807078_0018_m_000000_0' done.
2015-02-12 15:45:59,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1388807078_0018_m_000000_0
2015-02-12 15:45:59,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:59,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:59,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1388807078_0018_r_000000_0
2015-02-12 15:45:59,301 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,301 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,301 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4fedbe16
2015-02-12 15:45:59,302 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:59,302 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1388807078_0018_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:59,303 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#18 about to shuffle output of map attempt_local1388807078_0018_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-12 15:45:59,303 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1388807078_0018_m_000000_0
2015-02-12 15:45:59,303 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-12 15:45:59,304 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:59,304 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,304 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:59,306 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,306 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:59,307 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:59,308 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 709 bytes from disk
2015-02-12 15:45:59,308 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:59,308 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,308 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-12 15:45:59,309 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,309 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:59,309 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,313 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,314 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true"]
2015-02-12 15:45:59,324 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1388807078_0018_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,325 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,326 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1388807078_0018_r_000000_0 is allowed to commit now
2015-02-12 15:45:59,326 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1388807078_0018_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true/_temporary/0/task_local1388807078_0018_r_000000
2015-02-12 15:45:59,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:59,327 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1388807078_0018_r_000000_0' done.
2015-02-12 15:45:59,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1388807078_0018_r_000000_0
2015-02-12 15:45:59,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:59,347 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true/_temporary
2015-02-12 15:45:59,366 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,377 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:59,377 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:59,381 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:59,382 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:59,382 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:59,386 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:59,387 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,387 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true"]
2015-02-12 15:45:59,387 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:59,387 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:59,387 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:59,387 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_false_true
2015-02-12 15:45:59,388 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,388 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,407 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,426 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:59,438 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local279778327_0019
2015-02-12 15:45:59,448 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi279778327/.staging/job_local279778327_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,448 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi279778327/.staging/job_local279778327_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,473 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local279778327_0019/job_local279778327_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,474 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local279778327_0019/job_local279778327_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,475 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:59,475 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local279778327_0019
2015-02-12 15:45:59,475 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:59,475 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:59,475 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:59,477 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:59,477 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local279778327_0019_m_000000_0
2015-02-12 15:45:59,479 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,479 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,480 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:59,480 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@89df5e8
2015-02-12 15:45:59,481 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:59,501 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:59,501 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:59,501 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:59,501 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:59,502 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:59,503 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:59,503 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:59,503 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,509 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,509 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,512 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:59,512 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:59,512 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:59,512 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:59,512 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:59,517 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:59,518 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local279778327_0019_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,519 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:59,519 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local279778327_0019_m_000000_0' done.
2015-02-12 15:45:59,519 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local279778327_0019_m_000000_0
2015-02-12 15:45:59,519 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:59,520 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:59,520 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local279778327_0019_r_000000_0
2015-02-12 15:45:59,521 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,521 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,521 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1b9f8680
2015-02-12 15:45:59,522 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:59,522 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local279778327_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:59,523 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#19 about to shuffle output of map attempt_local279778327_0019_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:59,523 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local279778327_0019_m_000000_0
2015-02-12 15:45:59,523 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:59,524 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:59,524 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,524 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:59,526 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,526 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,528 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:59,528 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:59,528 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:59,529 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,529 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,530 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:59,530 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,534 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,534 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true"]
2015-02-12 15:45:59,544 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local279778327_0019_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,545 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,545 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local279778327_0019_r_000000_0 is allowed to commit now
2015-02-12 15:45:59,546 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local279778327_0019_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true/_temporary/0/task_local279778327_0019_r_000000
2015-02-12 15:45:59,546 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:59,546 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local279778327_0019_r_000000_0' done.
2015-02-12 15:45:59,546 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local279778327_0019_r_000000_0
2015-02-12 15:45:59,547 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:59,563 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true/_temporary
2015-02-12 15:45:59,583 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,593 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:59,593 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:59,598 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:59,599 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:59,600 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true"]
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:59,604 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:59,604 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_false_true
2015-02-12 15:45:59,605 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,606 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,629 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,649 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:59,661 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1039404348_0020
2015-02-12 15:45:59,669 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1039404348/.staging/job_local1039404348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,669 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1039404348/.staging/job_local1039404348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,696 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1039404348_0020/job_local1039404348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,697 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1039404348_0020/job_local1039404348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,698 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:59,698 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1039404348_0020
2015-02-12 15:45:59,698 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:59,698 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:59,698 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:59,700 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:59,700 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1039404348_0020_m_000000_0
2015-02-12 15:45:59,701 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,702 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,703 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:59,703 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@43f7cbeb
2015-02-12 15:45:59,704 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:59,721 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:59,721 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:59,721 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:59,721 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:59,721 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:59,722 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:59,722 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:59,723 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,727 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,728 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,731 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:59,731 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:59,731 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:59,731 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:59,731 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:59,733 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:59,735 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1039404348_0020_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:59,736 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1039404348_0020_m_000000_0' done.
2015-02-12 15:45:59,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1039404348_0020_m_000000_0
2015-02-12 15:45:59,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:59,737 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:59,737 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1039404348_0020_r_000000_0
2015-02-12 15:45:59,738 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,738 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,738 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b52d5ec
2015-02-12 15:45:59,739 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:59,740 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1039404348_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:59,741 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#20 about to shuffle output of map attempt_local1039404348_0020_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:59,742 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1039404348_0020_m_000000_0
2015-02-12 15:45:59,743 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:59,744 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:59,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,746 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:59,748 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,748 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,749 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:59,750 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:59,750 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:59,750 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,750 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,751 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,752 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:59,752 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,756 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,756 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true"]
2015-02-12 15:45:59,768 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1039404348_0020_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,769 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1039404348_0020_r_000000_0 is allowed to commit now
2015-02-12 15:45:59,770 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1039404348_0020_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true/_temporary/0/task_local1039404348_0020_r_000000
2015-02-12 15:45:59,770 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:59,771 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1039404348_0020_r_000000_0' done.
2015-02-12 15:45:59,771 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1039404348_0020_r_000000_0
2015-02-12 15:45:59,771 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:45:59,793 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true/_temporary
2015-02-12 15:45:59,812 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,822 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:45:59,822 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:45:59,826 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:45:59,827 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:45:59,827 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:45:59,831 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:45:59,831 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,831 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true"]
2015-02-12 15:45:59,832 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:45:59,832 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:45:59,832 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:45:59,832 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_false_true
2015-02-12 15:45:59,833 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,833 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:45:59,856 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:45:59,876 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:45:59,888 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1097331122_0021
2015-02-12 15:45:59,897 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1097331122/.staging/job_local1097331122_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,898 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1097331122/.staging/job_local1097331122_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,931 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1097331122_0021/job_local1097331122_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:45:59,932 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1097331122_0021/job_local1097331122_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:45:59,932 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:45:59,933 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:45:59,933 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1097331122_0021
2015-02-12 15:45:59,933 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:45:59,933 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:45:59,935 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:45:59,935 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1097331122_0021_m_000000_0
2015-02-12 15:45:59,936 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,936 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,937 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:45:59,937 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7dc31479
2015-02-12 15:45:59,937 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:45:59,953 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:45:59,953 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:45:59,953 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:45:59,953 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:45:59,953 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:45:59,955 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:45:59,955 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:45:59,955 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,959 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:45:59,959 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,963 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:45:59,963 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:45:59,963 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:45:59,963 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:45:59,963 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:45:59,966 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:45:59,968 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1097331122_0021_m_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,969 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:45:59,969 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1097331122_0021_m_000000_0' done.
2015-02-12 15:45:59,969 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1097331122_0021_m_000000_0
2015-02-12 15:45:59,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:45:59,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:45:59,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1097331122_0021_r_000000_0
2015-02-12 15:45:59,971 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:45:59,971 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:45:59,971 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2101516b
2015-02-12 15:45:59,972 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:45:59,973 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1097331122_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:45:59,974 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#21 about to shuffle output of map attempt_local1097331122_0021_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:45:59,974 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1097331122_0021_m_000000_0
2015-02-12 15:45:59,974 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:45:59,975 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:45:59,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,976 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:45:59,978 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,978 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,979 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:45:59,979 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:45:59,979 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:45:59,980 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:45:59,980 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:45:59,980 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,981 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:45:59,981 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:45:59,985 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:45:59,985 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true"]
2015-02-12 15:45:59,996 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1097331122_0021_r_000000_0 is done. And is in the process of committing
2015-02-12 15:45:59,997 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:45:59,997 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1097331122_0021_r_000000_0 is allowed to commit now
2015-02-12 15:45:59,998 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1097331122_0021_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true/_temporary/0/task_local1097331122_0021_r_000000
2015-02-12 15:45:59,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:45:59,999 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1097331122_0021_r_000000_0' done.
2015-02-12 15:45:59,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1097331122_0021_r_000000_0
2015-02-12 15:46:00,000 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:00,030 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true/_temporary
2015-02-12 15:46:00,050 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,060 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:46:00,061 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:46:00,064 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:46:00,066 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:46:00,066 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:46:00,070 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:46:00,071 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,071 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true"]
2015-02-12 15:46:00,071 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:46:00,071 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:46:00,071 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:46:00,071 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_false_true
2015-02-12 15:46:00,072 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,073 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,093 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,110 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:46:00,124 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1041907840_0022
2015-02-12 15:46:00,133 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1041907840/.staging/job_local1041907840_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,134 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1041907840/.staging/job_local1041907840_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,165 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1041907840_0022/job_local1041907840_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,166 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1041907840_0022/job_local1041907840_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,167 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:46:00,167 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1041907840_0022
2015-02-12 15:46:00,167 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:46:00,168 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:46:00,168 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:46:00,169 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:46:00,169 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1041907840_0022_m_000000_0
2015-02-12 15:46:00,170 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,170 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,171 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:46:00,171 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@55f53150
2015-02-12 15:46:00,172 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:46:00,188 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:46:00,188 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:46:00,188 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:46:00,188 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:46:00,188 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:46:00,189 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:46:00,189 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:46:00,189 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,195 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,195 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,198 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:46:00,198 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:46:00,198 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:46:00,198 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:46:00,198 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:46:00,201 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:46:00,202 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1041907840_0022_m_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:46:00,203 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1041907840_0022_m_000000_0' done.
2015-02-12 15:46:00,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1041907840_0022_m_000000_0
2015-02-12 15:46:00,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:46:00,204 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:46:00,204 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1041907840_0022_r_000000_0
2015-02-12 15:46:00,205 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,205 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,205 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7fa2de95
2015-02-12 15:46:00,205 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:46:00,206 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1041907840_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:46:00,207 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#22 about to shuffle output of map attempt_local1041907840_0022_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:46:00,207 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1041907840_0022_m_000000_0
2015-02-12 15:46:00,207 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:46:00,207 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:46:00,208 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,208 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:46:00,210 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,210 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,212 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:46:00,212 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:46:00,212 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:46:00,212 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,212 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,213 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,213 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:46:00,213 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,218 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,218 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true"]
2015-02-12 15:46:00,231 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1041907840_0022_r_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,232 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,232 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1041907840_0022_r_000000_0 is allowed to commit now
2015-02-12 15:46:00,232 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1041907840_0022_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true/_temporary/0/task_local1041907840_0022_r_000000
2015-02-12 15:46:00,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:46:00,233 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1041907840_0022_r_000000_0' done.
2015-02-12 15:46:00,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1041907840_0022_r_000000_0
2015-02-12 15:46:00,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:00,251 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true/_temporary
2015-02-12 15:46:00,278 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,288 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:46:00,288 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:46:00,292 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:46:00,294 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:46:00,295 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:46:00,301 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:46:00,302 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,302 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true"]
2015-02-12 15:46:00,302 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:46:00,302 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:46:00,302 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:46:00,304 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_false_true_true
2015-02-12 15:46:00,305 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,305 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,326 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,347 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:46:00,363 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local673731_0023
2015-02-12 15:46:00,373 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi673731/.staging/job_local673731_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,374 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi673731/.staging/job_local673731_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,400 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local673731_0023/job_local673731_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,402 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local673731_0023/job_local673731_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,403 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:46:00,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:46:00,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:46:00,404 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local673731_0023
2015-02-12 15:46:00,405 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:46:00,406 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:46:00,406 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local673731_0023_m_000000_0
2015-02-12 15:46:00,408 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,408 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,408 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:46:00,409 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4b911d5b
2015-02-12 15:46:00,409 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:46:00,425 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:46:00,425 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:46:00,426 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:46:00,426 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:46:00,426 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:46:00,427 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:46:00,427 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:46:00,427 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,431 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,431 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,434 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:46:00,434 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:46:00,434 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:46:00,435 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:46:00,435 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:46:00,437 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:46:00,439 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local673731_0023_m_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,441 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:46:00,441 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local673731_0023_m_000000_0' done.
2015-02-12 15:46:00,441 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local673731_0023_m_000000_0
2015-02-12 15:46:00,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:46:00,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:46:00,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local673731_0023_r_000000_0
2015-02-12 15:46:00,443 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,444 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,444 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d3ff748
2015-02-12 15:46:00,444 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:46:00,445 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local673731_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:46:00,445 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#23 about to shuffle output of map attempt_local673731_0023_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:46:00,446 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local673731_0023_m_000000_0
2015-02-12 15:46:00,446 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:46:00,446 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:46:00,447 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,447 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:46:00,449 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,449 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,450 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:46:00,450 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:46:00,450 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:46:00,450 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,451 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,451 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,451 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:46:00,451 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,455 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,455 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true"]
2015-02-12 15:46:00,465 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local673731_0023_r_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,466 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,466 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local673731_0023_r_000000_0 is allowed to commit now
2015-02-12 15:46:00,467 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local673731_0023_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true/_temporary/0/task_local673731_0023_r_000000
2015-02-12 15:46:00,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:46:00,468 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local673731_0023_r_000000_0' done.
2015-02-12 15:46:00,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local673731_0023_r_000000_0
2015-02-12 15:46:00,469 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:00,487 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true/_temporary
2015-02-12 15:46:00,505 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,517 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:46:00,517 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:46:00,520 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:46:00,522 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:46:00,523 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:46:00,527 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:46:00,527 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,527 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true"]
2015-02-12 15:46:00,527 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:46:00,528 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:46:00,528 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:46:00,528 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_false_true_true
2015-02-12 15:46:00,529 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,530 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,548 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,565 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:46:00,580 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local455067312_0024
2015-02-12 15:46:00,588 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi455067312/.staging/job_local455067312_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,588 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi455067312/.staging/job_local455067312_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,615 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local455067312_0024/job_local455067312_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,616 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local455067312_0024/job_local455067312_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,616 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:46:00,616 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local455067312_0024
2015-02-12 15:46:00,616 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:46:00,617 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:46:00,616 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:46:00,618 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:46:00,618 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local455067312_0024_m_000000_0
2015-02-12 15:46:00,619 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,619 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,620 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:46:00,620 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7795503d
2015-02-12 15:46:00,621 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:46:00,638 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:46:00,638 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:46:00,638 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:46:00,638 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:46:00,638 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:46:00,639 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:46:00,639 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:46:00,639 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,643 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,643 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,645 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:46:00,645 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:46:00,646 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:46:00,646 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:46:00,646 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:46:00,648 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:46:00,650 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local455067312_0024_m_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,651 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:46:00,651 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local455067312_0024_m_000000_0' done.
2015-02-12 15:46:00,651 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local455067312_0024_m_000000_0
2015-02-12 15:46:00,652 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:46:00,652 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:46:00,652 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local455067312_0024_r_000000_0
2015-02-12 15:46:00,654 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,654 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,654 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42b3fc2e
2015-02-12 15:46:00,654 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:46:00,655 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local455067312_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:46:00,656 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#24 about to shuffle output of map attempt_local455067312_0024_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:46:00,656 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local455067312_0024_m_000000_0
2015-02-12 15:46:00,656 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:46:00,657 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:46:00,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,657 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:46:00,659 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,660 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,661 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:46:00,662 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:46:00,662 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:46:00,662 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,662 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,662 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,663 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:46:00,663 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,668 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,668 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true"]
2015-02-12 15:46:00,682 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local455067312_0024_r_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,683 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,683 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local455067312_0024_r_000000_0 is allowed to commit now
2015-02-12 15:46:00,684 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local455067312_0024_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true/_temporary/0/task_local455067312_0024_r_000000
2015-02-12 15:46:00,685 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:46:00,685 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local455067312_0024_r_000000_0' done.
2015-02-12 15:46:00,685 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local455067312_0024_r_000000_0
2015-02-12 15:46:00,685 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:00,706 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true/_temporary
2015-02-12 15:46:00,728 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,738 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:46:00,738 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:46:00,742 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:46:00,743 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:46:00,743 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true"]
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:46:00,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:46:00,749 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...nsive/true_true_true_true
2015-02-12 15:46:00,750 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,750 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,770 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,786 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:46:00,798 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local376305697_0025
2015-02-12 15:46:00,806 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi376305697/.staging/job_local376305697_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,807 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi376305697/.staging/job_local376305697_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,833 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local376305697_0025/job_local376305697_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:00,834 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local376305697_0025/job_local376305697_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:00,834 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:46:00,834 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local376305697_0025
2015-02-12 15:46:00,834 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:46:00,834 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:46:00,835 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:46:00,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:46:00,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local376305697_0025_m_000000_0
2015-02-12 15:46:00,837 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,837 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,838 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:46:00,838 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b85ea56
2015-02-12 15:46:00,839 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:46:00,854 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:46:00,854 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:46:00,854 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:46:00,854 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:46:00,854 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:46:00,856 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:46:00,856 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:46:00,856 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,860 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,860 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,863 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:46:00,864 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:46:00,864 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:46:00,864 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:46:00,864 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:46:00,866 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:46:00,867 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local376305697_0025_m_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,869 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:46:00,869 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local376305697_0025_m_000000_0' done.
2015-02-12 15:46:00,869 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local376305697_0025_m_000000_0
2015-02-12 15:46:00,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:46:00,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:46:00,871 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local376305697_0025_r_000000_0
2015-02-12 15:46:00,872 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:00,873 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:00,873 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@44e32e90
2015-02-12 15:46:00,880 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:46:00,881 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local376305697_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:46:00,882 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#25 about to shuffle output of map attempt_local376305697_0025_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:46:00,882 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local376305697_0025_m_000000_0
2015-02-12 15:46:00,883 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:46:00,884 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:46:00,884 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,885 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:46:00,887 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,887 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,888 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:46:00,888 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:46:00,889 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:46:00,889 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:00,889 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:00,890 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,890 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:46:00,890 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:46:00,894 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:00,894 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true"]
2015-02-12 15:46:00,905 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local376305697_0025_r_000000_0 is done. And is in the process of committing
2015-02-12 15:46:00,906 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:00,906 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local376305697_0025_r_000000_0 is allowed to commit now
2015-02-12 15:46:00,906 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local376305697_0025_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true/_temporary/0/task_local376305697_0025_r_000000
2015-02-12 15:46:00,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:46:00,907 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local376305697_0025_r_000000_0' done.
2015-02-12 15:46:00,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local376305697_0025_r_000000_0
2015-02-12 15:46:00,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:00,927 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true/_temporary
2015-02-12 15:46:00,946 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:00,955 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 15:46:00,955 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 15:46:00,958 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 15:46:00,959 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 15:46:00,960 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 15:46:00,964 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-12 15:46:00,965 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:00,965 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true"]
2015-02-12 15:46:00,965 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-12 15:46:00,965 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-12 15:46:00,965 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-12 15:46:00,966 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/false_true_true_true
2015-02-12 15:46:00,968 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,969 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 15:46:00,989 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:01,008 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 15:46:01,020 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local761537979_0026
2015-02-12 15:46:01,028 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi761537979/.staging/job_local761537979_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:01,029 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi761537979/.staging/job_local761537979_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:01,060 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local761537979_0026/job_local761537979_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 15:46:01,062 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local761537979_0026/job_local761537979_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 15:46:01,062 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 15:46:01,062 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 15:46:01,062 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 15:46:01,062 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local761537979_0026
2015-02-12 15:46:01,063 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-12 15:46:01,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 15:46:01,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local761537979_0026_m_000000_0
2015-02-12 15:46:01,066 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:01,066 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:01,067 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt
2015-02-12 15:46:01,067 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@19343b59
2015-02-12 15:46:01,067 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 15:46:01,082 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 15:46:01,082 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 15:46:01,082 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 15:46:01,082 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 15:46:01,082 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 15:46:01,083 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 15:46:01,083 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 15:46:01,083 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 15:46:01,088 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-12 15:46:01,088 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:01,093 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 15:46:01,093 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 15:46:01,093 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 15:46:01,093 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-12 15:46:01,093 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-12 15:46:01,096 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 15:46:01,098 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local761537979_0026_m_000000_0 is done. And is in the process of committing
2015-02-12 15:46:01,099 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-12 15:46:01,100 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local761537979_0026_m_000000_0' done.
2015-02-12 15:46:01,100 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local761537979_0026_m_000000_0
2015-02-12 15:46:01,100 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 15:46:01,101 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 15:46:01,101 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local761537979_0026_r_000000_0
2015-02-12 15:46:01,103 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 15:46:01,103 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 15:46:01,103 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@b671b66
2015-02-12 15:46:01,104 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 15:46:01,105 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local761537979_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 15:46:01,106 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#26 about to shuffle output of map attempt_local761537979_0026_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-12 15:46:01,106 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local761537979_0026_m_000000_0
2015-02-12 15:46:01,107 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-12 15:46:01,108 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 15:46:01,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:01,109 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 15:46:01,112 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:01,112 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:01,113 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-12 15:46:01,114 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 968 bytes from disk
2015-02-12 15:46:01,114 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 15:46:01,114 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 15:46:01,114 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-12 15:46:01,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:01,115 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 15:46:01,115 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 15:46:01,120 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-12 15:46:01,120 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true"]
2015-02-12 15:46:01,135 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local761537979_0026_r_000000_0 is done. And is in the process of committing
2015-02-12 15:46:01,136 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 15:46:01,136 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local761537979_0026_r_000000_0 is allowed to commit now
2015-02-12 15:46:01,137 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local761537979_0026_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true/_temporary/0/task_local761537979_0026_r_000000
2015-02-12 15:46:01,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 15:46:01,138 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local761537979_0026_r_000000_0' done.
2015-02-12 15:46:01,139 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local761537979_0026_r_000000_0
2015-02-12 15:46:01,139 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 15:46:01,154 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true/_temporary
2015-02-12 15:46:01,176 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 15:46:01,178 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false
2015-02-12 15:46:01,178 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true
2015-02-12 15:46:01,179 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false
2015-02-12 15:46:01,179 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false
2015-02-12 15:46:01,179 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true
2015-02-12 15:46:01,179 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false
2015-02-12 15:46:01,180 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true
2015-02-12 15:46:01,180 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false
2015-02-12 15:46:01,180 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false
2015-02-12 15:46:01,180 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true
2015-02-12 15:46:01,180 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true
2015-02-12 15:46:01,181 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true
2015-02-12 15:46:01,182 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true
2015-02-12 15:46:01,183 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false