2014-04-10 10:57:58,040 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2014-04-10 10:57:58,068 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2014-04-10 10:57:58,069 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.BasicPipesPlatformTest
2014-04-10 10:57:58,081 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(106)) - not using cluster
2014-04-10 10:57:58,366 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-04-10 10:57:58,495 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:57:58,496 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:57:58,502 INFO property.AppProps (AppProps.java:getAppID(162)) - using app.id: D9E672EB184D41EBB709E608D4CE1F75
2014-04-10 10:57:58,539 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:57:58,555 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:57:58,591 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:57:58,619 INFO util.Version (Version.java:printBanner(78)) - Concurrent, Inc - Cascading 2.5.3
2014-04-10 10:57:58,620 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:57:58,621 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:57:58,621 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplechain"]
2014-04-10 10:57:58,621 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:57:58,622 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 2
2014-04-10 10:57:58,622 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:57:58,623 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/2)
2014-04-10 10:57:58,634 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2014-04-10 10:57:58,635 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-04-10 10:57:58,647 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:58,798 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:58,820 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:57:58,913 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2106117179_0001
2014-04-10 10:57:58,933 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2106117179/.staging/job_local2106117179_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:58,933 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock2106117179/.staging/job_local2106117179_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,032 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2106117179_0001/job_local2106117179_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,032 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local2106117179_0001/job_local2106117179_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,036 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:57:59,036 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:57:59,037 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local2106117179_0001
2014-04-10 10:57:59,037 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:57:59,037 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:57:59,064 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:57:59,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2106117179_0001_m_000000_0
2014-04-10 10:57:59,082 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,092 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:57:59,092 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@25d39e7d
2014-04-10 10:57:59,097 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:57:59,100 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:57:59,112 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:57:59,112 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:57:59,112 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:57:59,112 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:57:59,113 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:57:59,122 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:57:59,122 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,230 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2014-04-10 10:57:59,240 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:57:59,240 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:57:59,247 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:57:59,247 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:57:59,247 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:57:59,247 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2014-04-10 10:57:59,247 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2014-04-10 10:57:59,252 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:57:59,254 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2106117179_0001_m_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,259 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:57:59,259 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2106117179_0001_m_000000_0' done.
2014-04-10 10:57:59,259 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2106117179_0001_m_000000_0
2014-04-10 10:57:59,260 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:57:59,261 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:57:59,261 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2106117179_0001_r_000000_0
2014-04-10 10:57:59,265 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,266 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@419e8639
2014-04-10 10:57:59,273 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:57:59,274 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2106117179_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:57:59,289 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local2106117179_0001_m_000000_0 decomp: 232 len: 236 to MEMORY
2014-04-10 10:57:59,291 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 232 bytes from map-output for attempt_local2106117179_0001_m_000000_0
2014-04-10 10:57:59,306 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 232, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->232
2014-04-10 10:57:59,307 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:57:59,308 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,309 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:57:59,313 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,313 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:57:59,314 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 232 bytes to disk to satisfy reduce memory limit
2014-04-10 10:57:59,314 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 236 bytes from disk
2014-04-10 10:57:59,315 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:57:59,315 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,315 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:57:59,316 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,316 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:57:59,316 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,325 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:57:59,325 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[[0, 'count']]"][815197415/test/]
2014-04-10 10:57:59,343 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2106117179_0001_r_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,344 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,344 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2106117179_0001_r_000000_0 is allowed to commit now
2014-04-10 10:57:59,344 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2106117179_0001_r_000000_0' to file:/tmp/hadoop-brock/815197415_test_521ED53E294E4945B6E1F8E5E4104487/_temporary/0/task_local2106117179_0001_r_000000
2014-04-10 10:57:59,345 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:57:59,345 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2106117179_0001_r_000000_0' done.
2014-04-10 10:57:59,345 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2106117179_0001_r_000000_0
2014-04-10 10:57:59,345 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:57:59,367 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (2/2) ...pipesplatform/simplechain
2014-04-10 10:57:59,367 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,368 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,390 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,404 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:57:59,414 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:57:59,414 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:57:59,424 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1804170821_0002
2014-04-10 10:57:59,434 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1804170821/.staging/job_local1804170821_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,434 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1804170821/.staging/job_local1804170821_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,476 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1804170821_0002/job_local1804170821_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,477 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1804170821_0002/job_local1804170821_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,478 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:57:59,478 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:57:59,478 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1804170821_0002
2014-04-10 10:57:59,478 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:57:59,478 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:57:59,480 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:57:59,480 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1804170821_0002_m_000000_0
2014-04-10 10:57:59,481 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,482 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-brock/815197415_test_521ED53E294E4945B6E1F8E5E4104487/part-00000
2014-04-10 10:57:59,482 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2ede611c
2014-04-10 10:57:59,488 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:57:59,488 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:57:59,499 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:57:59,499 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:57:59,499 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:57:59,499 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:57:59,499 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:57:59,500 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:57:59,500 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,507 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[[0, 'count']]"][815197415/test/]
2014-04-10 10:57:59,507 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:57:59,508 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:57:59,509 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:57:59,509 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:57:59,509 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 184; bufvoid = 104857600
2014-04-10 10:57:59,509 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214368(104857472); length = 29/6553600
2014-04-10 10:57:59,511 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:57:59,512 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1804170821_0002_m_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-brock/815197415_test_521ED53E294E4945B6E1F8E5E4104487/part-00000:0+310
2014-04-10 10:57:59,514 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1804170821_0002_m_000000_0' done.
2014-04-10 10:57:59,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1804170821_0002_m_000000_0
2014-04-10 10:57:59,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:57:59,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:57:59,515 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1804170821_0002_r_000000_0
2014-04-10 10:57:59,517 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,517 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7778dc8e
2014-04-10 10:57:59,518 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:57:59,518 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1804170821_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:57:59,519 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local1804170821_0002_m_000000_0 decomp: 202 len: 206 to MEMORY
2014-04-10 10:57:59,520 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 202 bytes from map-output for attempt_local1804170821_0002_m_000000_0
2014-04-10 10:57:59,520 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 202, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->202
2014-04-10 10:57:59,520 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:57:59,521 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,521 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:57:59,524 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,524 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2014-04-10 10:57:59,526 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 202 bytes to disk to satisfy reduce memory limit
2014-04-10 10:57:59,526 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 206 bytes from disk
2014-04-10 10:57:59,527 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:57:59,527 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,527 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2014-04-10 10:57:59,528 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,529 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:57:59,529 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,534 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:57:59,534 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplechain"]
2014-04-10 10:57:59,543 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1804170821_0002_r_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,545 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,545 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1804170821_0002_r_000000_0 is allowed to commit now
2014-04-10 10:57:59,546 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1804170821_0002_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplechain/_temporary/0/task_local1804170821_0002_r_000000
2014-04-10 10:57:59,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:57:59,548 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1804170821_0002_r_000000_0' done.
2014-04-10 10:57:59,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1804170821_0002_r_000000_0
2014-04-10 10:57:59,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:57:59,565 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplechain/_temporary
2014-04-10 10:57:59,583 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,585 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/simplechain
2014-04-10 10:57:59,599 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:57:59,600 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:57:59,607 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:57:59,610 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:57:59,610 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count"]
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-04-10 10:57:59,616 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-04-10 10:57:59,617 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) .../basicpipesplatform/count
2014-04-10 10:57:59,617 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,618 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,629 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,636 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:57:59,657 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local335690906_0003
2014-04-10 10:57:59,669 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock335690906/.staging/job_local335690906_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,669 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock335690906/.staging/job_local335690906_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,702 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local335690906_0003/job_local335690906_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,702 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local335690906_0003/job_local335690906_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,703 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:57:59,703 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:57:59,703 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local335690906_0003
2014-04-10 10:57:59,703 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:57:59,703 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 10:57:59,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:57:59,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local335690906_0003_m_000000_0
2014-04-10 10:57:59,706 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,707 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt
2014-04-10 10:57:59,707 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@329e26c0
2014-04-10 10:57:59,708 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:57:59,708 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:57:59,719 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:57:59,719 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:57:59,719 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:57:59,719 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:57:59,719 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:57:59,721 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:57:59,721 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,726 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:57:59,726 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:57:59,728 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:57:59,728 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:57:59,728 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:57:59,728 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2014-04-10 10:57:59,728 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2014-04-10 10:57:59,731 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:57:59,733 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local335690906_0003_m_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,734 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt:0+275
2014-04-10 10:57:59,734 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local335690906_0003_m_000000_0' done.
2014-04-10 10:57:59,734 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local335690906_0003_m_000000_0
2014-04-10 10:57:59,734 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:57:59,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:57:59,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local335690906_0003_r_000000_0
2014-04-10 10:57:59,736 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,737 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7f2175f0
2014-04-10 10:57:59,737 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:57:59,737 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local335690906_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:57:59,738 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local335690906_0003_m_000000_0 decomp: 508 len: 512 to MEMORY
2014-04-10 10:57:59,739 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 508 bytes from map-output for attempt_local335690906_0003_m_000000_0
2014-04-10 10:57:59,739 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 508, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->508
2014-04-10 10:57:59,739 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:57:59,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,740 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:57:59,741 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,741 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:57:59,742 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 508 bytes to disk to satisfy reduce memory limit
2014-04-10 10:57:59,742 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 512 bytes from disk
2014-04-10 10:57:59,742 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:57:59,742 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:57:59,743 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:57:59,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,743 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:57:59,744 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,749 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:57:59,749 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count"]
2014-04-10 10:57:59,758 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local335690906_0003_r_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,759 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:57:59,759 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local335690906_0003_r_000000_0 is allowed to commit now
2014-04-10 10:57:59,760 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local335690906_0003_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count/_temporary/0/task_local335690906_0003_r_000000
2014-04-10 10:57:59,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:57:59,761 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local335690906_0003_r_000000_0' done.
2014-04-10 10:57:59,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local335690906_0003_r_000000_0
2014-04-10 10:57:59,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:57:59,778 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count/_temporary
2014-04-10 10:57:59,796 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,813 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,815 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/count
2014-04-10 10:57:59,828 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:57:59,829 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:57:59,834 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:57:59,837 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:57:59,837 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/swap"]
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:57:59,842 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:57:59,843 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...1/basicpipesplatform/swap
2014-04-10 10:57:59,843 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,844 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:57:59,892 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:57:59,898 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:57:59,908 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local33552209_0004
2014-04-10 10:57:59,919 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock33552209/.staging/job_local33552209_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,919 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock33552209/.staging/job_local33552209_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,948 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local33552209_0004/job_local33552209_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:57:59,948 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local33552209_0004/job_local33552209_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:57:59,949 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:57:59,949 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local33552209_0004
2014-04-10 10:57:59,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:57:59,949 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:57:59,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:57:59,952 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:57:59,952 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local33552209_0004_m_000000_0
2014-04-10 10:57:59,953 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:57:59,954 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:57:59,954 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5be6f926
2014-04-10 10:57:59,955 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 10:57:59,959 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:57:59,959 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:57:59,963 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:57:59,963 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/swap"]
2014-04-10 10:57:59,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:57:59,965 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local33552209_0004_m_000000_0 is done. And is in the process of committing
2014-04-10 10:57:59,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:57:59,966 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local33552209_0004_m_000000_0 is allowed to commit now
2014-04-10 10:57:59,967 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local33552209_0004_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/swap/_temporary/0/task_local33552209_0004_m_000000
2014-04-10 10:57:59,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:57:59,967 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local33552209_0004_m_000000_0' done.
2014-04-10 10:57:59,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local33552209_0004_m_000000_0
2014-04-10 10:57:59,968 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:57:59,982 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/swap/_temporary
2014-04-10 10:58:00,005 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,006 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/swap
2014-04-10 10:58:00,021 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:00,021 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:00,027 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:00,029 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:00,030 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:00,035 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:00,035 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower+upper.txt"]
2014-04-10 10:58:00,036 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/ungrouped"]
2014-04-10 10:58:00,036 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:00,036 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:00,036 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:00,036 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...icpipesplatform/ungrouped
2014-04-10 10:58:00,036 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,037 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,046 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,053 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:00,072 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local774190803_0005
2014-04-10 10:58:00,081 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock774190803/.staging/job_local774190803_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,081 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock774190803/.staging/job_local774190803_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local774190803_0005/job_local774190803_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local774190803_0005/job_local774190803_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,115 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:00,115 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local774190803_0005
2014-04-10 10:58:00,115 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:00,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:00,116 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:00,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:00,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local774190803_0005_m_000000_0
2014-04-10 10:58:00,120 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,120 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower+upper.txt
2014-04-10 10:58:00,120 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7c9416f5
2014-04-10 10:58:00,121 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 10:58:00,125 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:00,125 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,130 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower+upper.txt"]
2014-04-10 10:58:00,131 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/ungrouped"]
2014-04-10 10:58:00,134 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,135 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local774190803_0005_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,137 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,137 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local774190803_0005_m_000000_0 is allowed to commit now
2014-04-10 10:58:00,137 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local774190803_0005_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/ungrouped/_temporary/0/task_local774190803_0005_m_000000
2014-04-10 10:58:00,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower+upper.txt:0+30
2014-04-10 10:58:00,138 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local774190803_0005_m_000000_0' done.
2014-04-10 10:58:00,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local774190803_0005_m_000000_0
2014-04-10 10:58:00,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:00,159 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/ungrouped/_temporary
2014-04-10 10:58:00,173 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,174 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/ungrouped
2014-04-10 10:58:00,183 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:00,184 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:00,190 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:00,192 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:00,192 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:00,197 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:00,197 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:00,198 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplerelative"]
2014-04-10 10:58:00,198 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:00,198 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:00,198 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:00,198 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...esplatform/simplerelative
2014-04-10 10:58:00,199 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,200 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,213 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,224 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:00,237 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1160295242_0006
2014-04-10 10:58:00,252 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1160295242/.staging/job_local1160295242_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,252 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1160295242/.staging/job_local1160295242_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,285 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1160295242_0006/job_local1160295242_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,285 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1160295242_0006/job_local1160295242_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,285 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:00,285 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1160295242_0006
2014-04-10 10:58:00,286 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:00,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:00,286 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:00,289 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:00,289 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1160295242_0006_m_000000_0
2014-04-10 10:58:00,290 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,291 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:58:00,291 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1c62d0ce
2014-04-10 10:58:00,292 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:00,292 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:00,302 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:00,302 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:00,302 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:00,302 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:00,302 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:00,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:00,303 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:00,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:00,309 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,310 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:00,310 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:00,310 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2014-04-10 10:58:00,310 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2014-04-10 10:58:00,311 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:00,312 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1160295242_0006_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:58:00,314 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1160295242_0006_m_000000_0' done.
2014-04-10 10:58:00,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1160295242_0006_m_000000_0
2014-04-10 10:58:00,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:00,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:00,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1160295242_0006_r_000000_0
2014-04-10 10:58:00,316 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,316 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@608916f9
2014-04-10 10:58:00,316 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:00,316 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1160295242_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:00,318 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local1160295242_0006_m_000000_0 decomp: 232 len: 236 to MEMORY
2014-04-10 10:58:00,319 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 232 bytes from map-output for attempt_local1160295242_0006_m_000000_0
2014-04-10 10:58:00,319 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 232, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->232
2014-04-10 10:58:00,320 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:00,320 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,320 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:00,321 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:00,322 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:58:00,322 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 232 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:00,323 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 236 bytes from disk
2014-04-10 10:58:00,323 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:00,323 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:00,323 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:58:00,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,324 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:00,324 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,329 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:00,329 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplerelative"]
2014-04-10 10:58:00,338 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1160295242_0006_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,339 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,339 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1160295242_0006_r_000000_0 is allowed to commit now
2014-04-10 10:58:00,340 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1160295242_0006_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplerelative/_temporary/0/task_local1160295242_0006_r_000000
2014-04-10 10:58:00,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:00,341 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1160295242_0006_r_000000_0' done.
2014-04-10 10:58:00,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1160295242_0006_r_000000_0
2014-04-10 10:58:00,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:00,363 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simplerelative/_temporary
2014-04-10 10:58:00,377 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,379 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/simplerelative
2014-04-10 10:58:00,457 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:00,458 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:00,464 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:00,465 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:00,466 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:00,474 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:00,474 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 10:58:00,475 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simpleresult"]
2014-04-10 10:58:00,475 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:00,475 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:00,475 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:00,475 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...ipesplatform/simpleresult
2014-04-10 10:58:00,476 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,477 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,494 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,507 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:00,520 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1591493615_0007
2014-04-10 10:58:00,534 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1591493615/.staging/job_local1591493615_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,534 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1591493615/.staging/job_local1591493615_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,559 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1591493615_0007/job_local1591493615_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,559 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1591493615_0007/job_local1591493615_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,560 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:00,560 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:00,560 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1591493615_0007
2014-04-10 10:58:00,560 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:00,560 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:00,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:00,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1591493615_0007_m_000000_0
2014-04-10 10:58:00,565 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,566 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 10:58:00,567 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@62d7e0f7
2014-04-10 10:58:00,568 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:00,568 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:00,581 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:00,582 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:00,582 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:00,582 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:00,582 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:00,582 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:00,583 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,590 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 10:58:00,590 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:00,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,638 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:00,638 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:00,638 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2014-04-10 10:58:00,638 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2014-04-10 10:58:00,640 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:00,641 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1591493615_0007_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 10:58:00,642 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1591493615_0007_m_000000_0' done.
2014-04-10 10:58:00,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1591493615_0007_m_000000_0
2014-04-10 10:58:00,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:00,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:00,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1591493615_0007_r_000000_0
2014-04-10 10:58:00,644 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,644 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@643f2acd
2014-04-10 10:58:00,644 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:00,645 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1591493615_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:00,646 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local1591493615_0007_m_000000_0 decomp: 28 len: 32 to MEMORY
2014-04-10 10:58:00,647 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 28 bytes from map-output for attempt_local1591493615_0007_m_000000_0
2014-04-10 10:58:00,647 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 28, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->28
2014-04-10 10:58:00,647 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:00,648 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,648 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:00,649 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:00,649 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2014-04-10 10:58:00,650 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 28 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:00,650 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 32 bytes from disk
2014-04-10 10:58:00,650 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:00,650 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:00,651 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2014-04-10 10:58:00,651 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,651 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:00,651 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,659 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:00,659 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simpleresult"]
2014-04-10 10:58:00,665 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1591493615_0007_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,667 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:00,667 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1591493615_0007_r_000000_0 is allowed to commit now
2014-04-10 10:58:00,668 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1591493615_0007_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simpleresult/_temporary/0/task_local1591493615_0007_r_000000
2014-04-10 10:58:00,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:00,669 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1591493615_0007_r_000000_0' done.
2014-04-10 10:58:00,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1591493615_0007_r_000000_0
2014-04-10 10:58:00,670 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:00,690 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simpleresult/_temporary
2014-04-10 10:58:00,708 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,710 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/simpleresult
2014-04-10 10:58:00,724 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:00,724 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:00,728 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:00,729 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:00,730 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:00,735 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:00,736 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:00,736 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filterall"]
2014-04-10 10:58:00,737 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:00,737 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:00,737 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:00,738 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...icpipesplatform/filterall
2014-04-10 10:58:00,738 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,739 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,750 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,757 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:00,764 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1335543304_0008
2014-04-10 10:58:00,772 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1335543304/.staging/job_local1335543304_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,773 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1335543304/.staging/job_local1335543304_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,800 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1335543304_0008/job_local1335543304_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,800 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1335543304_0008/job_local1335543304_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,801 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:00,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:00,801 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1335543304_0008
2014-04-10 10:58:00,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:00,801 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:00,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:00,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1335543304_0008_m_000000_0
2014-04-10 10:58:00,803 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,804 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:58:00,804 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@27d5dfac
2014-04-10 10:58:00,805 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 10:58:00,808 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:00,808 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:00,812 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:00,812 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filterall"]
2014-04-10 10:58:00,813 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,813 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1335543304_0008_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:00,814 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:00,814 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1335543304_0008_m_000000_0 is allowed to commit now
2014-04-10 10:58:00,815 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1335543304_0008_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filterall/_temporary/0/task_local1335543304_0008_m_000000
2014-04-10 10:58:00,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:58:00,816 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1335543304_0008_m_000000_0' done.
2014-04-10 10:58:00,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1335543304_0008_m_000000_0
2014-04-10 10:58:00,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:00,834 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filterall/_temporary
2014-04-10 10:58:00,850 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,852 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/filterall
2014-04-10 10:58:00,870 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:00,871 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:00,877 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:00,880 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:00,880 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:00,887 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] starting
2014-04-10 10:58:00,887 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/upper.txt"]
2014-04-10 10:58:00,887 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 10:58:00,888 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/complexcogroup"]
2014-04-10 10:58:00,888 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] parallel execution is enabled: false
2014-04-10 10:58:00,888 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] starting jobs: 1
2014-04-10 10:58:00,888 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lower*upper] allocating threads: 1
2014-04-10 10:58:00,888 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lower*upper] starting step: (1/1) ...esplatform/complexcogroup
2014-04-10 10:58:00,889 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,889 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:00,906 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,909 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:00,921 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2014-04-10 10:58:00,937 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local460141062_0009
2014-04-10 10:58:00,954 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock460141062/.staging/job_local460141062_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,954 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock460141062/.staging/job_local460141062_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,984 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local460141062_0009/job_local460141062_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:00,984 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local460141062_0009/job_local460141062_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:00,985 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:00,985 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lower*upper] submitted hadoop job: job_local460141062_0009
2014-04-10 10:58:00,985 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:00,985 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lower*upper] tracking url: http://localhost:8080/
2014-04-10 10:58:00,985 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:00,986 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:00,986 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local460141062_0009_m_000000_0
2014-04-10 10:58:00,988 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:00,989 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/upper.txt
2014-04-10 10:58:00,989 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@64ae8431
2014-04-10 10:58:00,990 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:00,990 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:01,000 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:01,001 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:01,001 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:01,001 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:01,001 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:01,002 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,002 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,007 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/upper.txt"]
2014-04-10 10:58:01,007 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2014-04-10 10:58:01,009 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,009 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:01,009 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:01,009 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 85; bufvoid = 104857600
2014-04-10 10:58:01,009 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 10:58:01,011 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:01,013 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local460141062_0009_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/upper.txt:0+19
2014-04-10 10:58:01,015 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local460141062_0009_m_000000_0' done.
2014-04-10 10:58:01,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local460141062_0009_m_000000_0
2014-04-10 10:58:01,016 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local460141062_0009_m_000001_0
2014-04-10 10:58:01,017 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,019 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt
2014-04-10 10:58:01,019 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@43c7208e
2014-04-10 10:58:01,020 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:01,020 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:01,030 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:01,030 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:01,030 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:01,030 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:01,030 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:01,031 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,031 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,035 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt"]
2014-04-10 10:58:01,035 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2014-04-10 10:58:01,037 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,037 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:01,037 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:01,037 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 85; bufvoid = 104857600
2014-04-10 10:58:01,037 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2014-04-10 10:58:01,038 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:01,039 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local460141062_0009_m_000001_0 is done. And is in the process of committing
2014-04-10 10:58:01,040 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/lower.txt:0+19
2014-04-10 10:58:01,040 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local460141062_0009_m_000001_0' done.
2014-04-10 10:58:01,040 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local460141062_0009_m_000001_0
2014-04-10 10:58:01,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:01,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:01,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local460141062_0009_r_000000_0
2014-04-10 10:58:01,044 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,044 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1e8e2fe7
2014-04-10 10:58:01,045 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:01,045 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local460141062_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:01,047 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local460141062_0009_m_000001_0 decomp: 97 len: 101 to MEMORY
2014-04-10 10:58:01,047 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 97 bytes from map-output for attempt_local460141062_0009_m_000001_0
2014-04-10 10:58:01,047 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 97, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->97
2014-04-10 10:58:01,050 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local460141062_0009_m_000000_0 decomp: 97 len: 101 to MEMORY
2014-04-10 10:58:01,050 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 97 bytes from map-output for attempt_local460141062_0009_m_000000_0
2014-04-10 10:58:01,050 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 97, inMemoryMapOutputs.size() -> 2, commitMemory -> 97, usedMemory ->194
2014-04-10 10:58:01,051 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:01,052 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 10:58:01,052 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:01,055 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2014-04-10 10:58:01,055 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2014-04-10 10:58:01,057 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:01,057 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2014-04-10 10:58:01,057 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:01,057 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,058 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2014-04-10 10:58:01,059 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 10:58:01,059 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:01,059 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,063 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2014-04-10 10:58:01,063 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/complexcogroup"]
2014-04-10 10:58:01,074 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 10:58:01,074 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2014-04-10 10:58:01,080 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local460141062_0009_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,081 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2014-04-10 10:58:01,081 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local460141062_0009_r_000000_0 is allowed to commit now
2014-04-10 10:58:01,081 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local460141062_0009_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/complexcogroup/_temporary/0/task_local460141062_0009_r_000000
2014-04-10 10:58:01,082 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:01,082 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local460141062_0009_r_000000_0' done.
2014-04-10 10:58:01,082 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local460141062_0009_r_000000_0
2014-04-10 10:58:01,083 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:01,103 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/complexcogroup/_temporary
2014-04-10 10:58:01,124 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,140 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,142 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/complexcogroup
2014-04-10 10:58:01,152 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:01,152 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:01,157 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:01,158 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:01,159 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:01,162 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:01,163 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:01,163 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/replace"]
2014-04-10 10:58:01,163 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:01,163 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:01,163 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:01,164 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...asicpipesplatform/replace
2014-04-10 10:58:01,164 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,166 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,175 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,181 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:01,187 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1043088885_0010
2014-04-10 10:58:01,196 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1043088885/.staging/job_local1043088885_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,196 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1043088885/.staging/job_local1043088885_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,224 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1043088885_0010/job_local1043088885_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,224 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1043088885_0010/job_local1043088885_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,225 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:01,225 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:01,225 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1043088885_0010
2014-04-10 10:58:01,225 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:01,225 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:01,227 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:01,227 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1043088885_0010_m_000000_0
2014-04-10 10:58:01,228 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,229 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:58:01,229 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@69248a85
2014-04-10 10:58:01,229 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 10:58:01,232 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,232 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,235 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:01,236 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/replace"]
2014-04-10 10:58:01,238 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,238 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1043088885_0010_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,239 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,240 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1043088885_0010_m_000000_0 is allowed to commit now
2014-04-10 10:58:01,240 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1043088885_0010_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/replace/_temporary/0/task_local1043088885_0010_m_000000
2014-04-10 10:58:01,241 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:58:01,241 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1043088885_0010_m_000000_0' done.
2014-04-10 10:58:01,241 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1043088885_0010_m_000000_0
2014-04-10 10:58:01,242 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:01,259 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/replace/_temporary
2014-04-10 10:58:01,275 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,276 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/replace
2014-04-10 10:58:01,285 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:01,285 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:01,289 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:01,290 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:01,291 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:01,294 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 10:58:01,295 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,295 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count2"]
2014-04-10 10:58:01,295 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-04-10 10:58:01,295 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-04-10 10:58:01,295 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-04-10 10:58:01,296 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...basicpipesplatform/count2
2014-04-10 10:58:01,296 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,296 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,305 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,310 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:01,316 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local938950687_0011
2014-04-10 10:58:01,325 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock938950687/.staging/job_local938950687_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,325 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock938950687/.staging/job_local938950687_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,347 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local938950687_0011/job_local938950687_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,347 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local938950687_0011/job_local938950687_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,348 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:01,348 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:01,348 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local938950687_0011
2014-04-10 10:58:01,348 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:01,348 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 10:58:01,349 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:01,349 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local938950687_0011_m_000000_0
2014-04-10 10:58:01,350 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,351 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt
2014-04-10 10:58:01,351 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6300fba5
2014-04-10 10:58:01,351 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:01,351 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:01,363 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:01,363 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:01,363 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:01,363 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:01,363 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:01,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,367 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,367 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,369 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,369 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:01,369 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:01,369 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2014-04-10 10:58:01,369 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2014-04-10 10:58:01,372 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:01,372 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local938950687_0011_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,373 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt:0+275
2014-04-10 10:58:01,373 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local938950687_0011_m_000000_0' done.
2014-04-10 10:58:01,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local938950687_0011_m_000000_0
2014-04-10 10:58:01,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:01,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:01,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local938950687_0011_r_000000_0
2014-04-10 10:58:01,375 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,375 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@457105b0
2014-04-10 10:58:01,375 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:01,376 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local938950687_0011_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:01,376 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local938950687_0011_m_000000_0 decomp: 508 len: 512 to MEMORY
2014-04-10 10:58:01,376 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 508 bytes from map-output for attempt_local938950687_0011_m_000000_0
2014-04-10 10:58:01,376 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 508, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->508
2014-04-10 10:58:01,377 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:01,377 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,377 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:01,378 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,378 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,379 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 508 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:01,379 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 512 bytes from disk
2014-04-10 10:58:01,379 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:01,379 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,379 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,380 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,380 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:01,380 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,383 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,383 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count2"]
2014-04-10 10:58:01,388 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local938950687_0011_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,389 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,389 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local938950687_0011_r_000000_0 is allowed to commit now
2014-04-10 10:58:01,389 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local938950687_0011_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count2/_temporary/0/task_local938950687_0011_r_000000
2014-04-10 10:58:01,390 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:01,390 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local938950687_0011_r_000000_0' done.
2014-04-10 10:58:01,390 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local938950687_0011_r_000000_0
2014-04-10 10:58:01,390 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:01,407 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count2/_temporary
2014-04-10 10:58:01,428 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,445 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,446 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/count2
2014-04-10 10:58:01,458 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:01,458 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:01,462 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:01,463 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:01,463 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:01,467 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 10:58:01,467 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,467 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count3"]
2014-04-10 10:58:01,467 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-04-10 10:58:01,468 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-04-10 10:58:01,468 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-04-10 10:58:01,468 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...basicpipesplatform/count3
2014-04-10 10:58:01,468 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,469 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,485 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,495 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:01,502 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local202262199_0012
2014-04-10 10:58:01,513 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock202262199/.staging/job_local202262199_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,513 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock202262199/.staging/job_local202262199_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,535 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local202262199_0012/job_local202262199_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,535 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local202262199_0012/job_local202262199_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,535 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:01,535 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local202262199_0012
2014-04-10 10:58:01,536 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:01,536 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 10:58:01,536 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:01,537 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:01,537 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local202262199_0012_m_000000_0
2014-04-10 10:58:01,538 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,539 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt
2014-04-10 10:58:01,539 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3a763d2f
2014-04-10 10:58:01,540 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:01,540 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:01,556 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:01,556 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:01,556 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:01,556 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:01,557 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:01,557 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,557 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,563 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:01,563 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:01,563 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2014-04-10 10:58:01,563 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2014-04-10 10:58:01,566 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:01,566 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local202262199_0012_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,567 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt:0+275
2014-04-10 10:58:01,568 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local202262199_0012_m_000000_0' done.
2014-04-10 10:58:01,568 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local202262199_0012_m_000000_0
2014-04-10 10:58:01,568 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:01,568 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:01,568 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local202262199_0012_r_000000_0
2014-04-10 10:58:01,570 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,570 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@221a359f
2014-04-10 10:58:01,570 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:01,571 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local202262199_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:01,572 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local202262199_0012_m_000000_0 decomp: 508 len: 512 to MEMORY
2014-04-10 10:58:01,573 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 508 bytes from map-output for attempt_local202262199_0012_m_000000_0
2014-04-10 10:58:01,573 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 508, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->508
2014-04-10 10:58:01,574 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:01,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,576 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:01,577 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,577 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,579 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 508 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:01,579 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 512 bytes from disk
2014-04-10 10:58:01,579 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:01,580 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,580 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,581 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,581 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:01,581 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,588 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,588 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count3"]
2014-04-10 10:58:01,597 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local202262199_0012_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,599 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local202262199_0012_r_000000_0 is allowed to commit now
2014-04-10 10:58:01,599 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local202262199_0012_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count3/_temporary/0/task_local202262199_0012_r_000000
2014-04-10 10:58:01,600 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:01,600 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local202262199_0012_r_000000_0' done.
2014-04-10 10:58:01,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local202262199_0012_r_000000_0
2014-04-10 10:58:01,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:01,625 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count3/_temporary
2014-04-10 10:58:01,639 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,665 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,667 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/count3
2014-04-10 10:58:01,682 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:01,682 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:01,686 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:01,687 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:01,687 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:01,691 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-04-10 10:58:01,691 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,691 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count4"]
2014-04-10 10:58:01,691 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-04-10 10:58:01,691 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-04-10 10:58:01,692 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-04-10 10:58:01,692 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...basicpipesplatform/count4
2014-04-10 10:58:01,693 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,694 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,710 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,721 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:01,732 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1091176704_0013
2014-04-10 10:58:01,744 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1091176704/.staging/job_local1091176704_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,744 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1091176704/.staging/job_local1091176704_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,771 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1091176704_0013/job_local1091176704_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,772 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1091176704_0013/job_local1091176704_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:01,772 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:01,773 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1091176704_0013
2014-04-10 10:58:01,773 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-04-10 10:58:01,773 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:01,773 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:01,776 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:01,776 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1091176704_0013_m_000000_0
2014-04-10 10:58:01,778 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,780 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt
2014-04-10 10:58:01,780 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@762d59a3
2014-04-10 10:58:01,781 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:01,781 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:01,796 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:01,798 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:01,798 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:01,798 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:01,799 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:01,801 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:01,801 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,804 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt"]
2014-04-10 10:58:01,804 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,806 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:01,806 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:01,806 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:01,806 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2014-04-10 10:58:01,806 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2014-04-10 10:58:01,808 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:01,809 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1091176704_0013_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,810 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/ips.20.txt:0+275
2014-04-10 10:58:01,810 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1091176704_0013_m_000000_0' done.
2014-04-10 10:58:01,810 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1091176704_0013_m_000000_0
2014-04-10 10:58:01,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:01,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:01,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1091176704_0013_r_000000_0
2014-04-10 10:58:01,813 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:01,813 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@27924b38
2014-04-10 10:58:01,814 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=496232032, maxSingleShuffleLimit=124058008, mergeThreshold=327513152, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:01,815 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1091176704_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:01,816 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local1091176704_0013_m_000000_0 decomp: 508 len: 512 to MEMORY
2014-04-10 10:58:01,817 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 508 bytes from map-output for attempt_local1091176704_0013_m_000000_0
2014-04-10 10:58:01,817 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 508, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->508
2014-04-10 10:58:01,818 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:01,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,819 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:01,821 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,821 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,823 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 508 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:01,823 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 512 bytes from disk
2014-04-10 10:58:01,823 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:01,824 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:01,824 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 487 bytes
2014-04-10 10:58:01,825 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,825 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:01,825 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:01,830 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:1]]
2014-04-10 10:58:01,830 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count4"]
2014-04-10 10:58:01,839 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1091176704_0013_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:01,841 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:01,841 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1091176704_0013_r_000000_0 is allowed to commit now
2014-04-10 10:58:01,841 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1091176704_0013_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count4/_temporary/0/task_local1091176704_0013_r_000000
2014-04-10 10:58:01,843 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:01,843 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1091176704_0013_r_000000_0' done.
2014-04-10 10:58:01,843 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1091176704_0013_r_000000_0
2014-04-10 10:58:01,843 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:01,869 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/count4/_temporary
2014-04-10 10:58:01,888 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,908 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,909 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/count4
2014-04-10 10:58:01,921 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:01,922 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:01,924 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:01,925 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:01,925 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:01,929 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:01,929 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:01,929 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filter"]
2014-04-10 10:58:01,930 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:01,930 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:01,930 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:01,930 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...basicpipesplatform/filter
2014-04-10 10:58:01,931 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,931 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:01,946 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:01,957 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:01,967 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1530088472_0014
2014-04-10 10:58:01,981 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1530088472/.staging/job_local1530088472_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:01,982 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock1530088472/.staging/job_local1530088472_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:02,011 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1530088472_0014/job_local1530088472_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:02,011 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local1530088472_0014/job_local1530088472_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:02,012 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:02,012 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1530088472_0014
2014-04-10 10:58:02,012 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:02,012 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:02,013 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:02,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:02,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1530088472_0014_m_000000_0
2014-04-10 10:58:02,016 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:02,016 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:58:02,016 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4ccca363
2014-04-10 10:58:02,017 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2014-04-10 10:58:02,019 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:02,019 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:02,025 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:02,026 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filter"]
2014-04-10 10:58:02,029 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:02,029 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1530088472_0014_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:02,030 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:02,030 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1530088472_0014_m_000000_0 is allowed to commit now
2014-04-10 10:58:02,030 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1530088472_0014_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filter/_temporary/0/task_local1530088472_0014_m_000000
2014-04-10 10:58:02,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:58:02,031 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1530088472_0014_m_000000_0' done.
2014-04-10 10:58:02,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1530088472_0014_m_000000_0
2014-04-10 10:58:02,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:02,047 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/filter/_temporary
2014-04-10 10:58:02,070 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:02,072 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/filter
2014-04-10 10:58:02,089 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-10 10:58:02,090 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.5.3/fbde5367797af408c100d25704a77959e50a31d8/cascading-hadoop2-mr1-2.5.3.jar
2014-04-10 10:58:02,095 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2014-04-10 10:58:02,096 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2014-04-10 10:58:02,097 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simple"]
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] parallel execution is enabled: false
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] starting jobs: 1
2014-04-10 10:58:02,101 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [test] allocating threads: 1
2014-04-10 10:58:02,102 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...basicpipesplatform/simple
2014-04-10 10:58:02,102 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:02,102 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-04-10 10:58:02,115 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:02,124 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2014-04-10 10:58:02,135 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local166760767_0015
2014-04-10 10:58:02,150 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock166760767/.staging/job_local166760767_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:02,150 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/staging/brock166760767/.staging/job_local166760767_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:02,170 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local166760767_0015/job_local166760767_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2014-04-10 10:58:02,170 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/tmp/mapred/local/localRunner/brock/job_local166760767_0015/job_local166760767_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2014-04-10 10:58:02,171 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2014-04-10 10:58:02,171 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local166760767_0015
2014-04-10 10:58:02,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2014-04-10 10:58:02,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2014-04-10 10:58:02,171 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2014-04-10 10:58:02,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2014-04-10 10:58:02,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local166760767_0015_m_000000_0
2014-04-10 10:58:02,174 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:02,174 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt
2014-04-10 10:58:02,175 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@407773d4
2014-04-10 10:58:02,175 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2014-04-10 10:58:02,175 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2014-04-10 10:58:02,189 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2014-04-10 10:58:02,190 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2014-04-10 10:58:02,190 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2014-04-10 10:58:02,190 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2014-04-10 10:58:02,191 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2014-04-10 10:58:02,193 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-10 10:58:02,193 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-10 10:58:02,197 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt"]
2014-04-10 10:58:02,197 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:02,199 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2014-04-10 10:58:02,199 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2014-04-10 10:58:02,199 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2014-04-10 10:58:02,199 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2014-04-10 10:58:02,199 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2014-04-10 10:58:02,201 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2014-04-10 10:58:02,201 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local166760767_0015_m_000000_0 is done. And is in the process of committing
2014-04-10 10:58:02,202 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/classes/test/data/apache.10.txt:0+1449
2014-04-10 10:58:02,202 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local166760767_0015_m_000000_0' done.
2014-04-10 10:58:02,202 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local166760767_0015_m_000000_0
2014-04-10 10:58:02,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2014-04-10 10:58:02,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2014-04-10 10:58:02,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local166760767_0015_r_000000_0
2014-04-10 10:58:02,204 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2014-04-10 10:58:02,204 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42393d16
2014-04-10 10:58:02,204 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=496920160, maxSingleShuffleLimit=124230040, mergeThreshold=327967328, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2014-04-10 10:58:02,204 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local166760767_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2014-04-10 10:58:02,205 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local166760767_0015_m_000000_0 decomp: 232 len: 236 to MEMORY
2014-04-10 10:58:02,205 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 232 bytes from map-output for attempt_local166760767_0015_m_000000_0
2014-04-10 10:58:02,205 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 232, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->232
2014-04-10 10:58:02,206 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2014-04-10 10:58:02,206 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:02,207 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2014-04-10 10:58:02,207 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:02,207 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:58:02,208 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 232 bytes to disk to satisfy reduce memory limit
2014-04-10 10:58:02,208 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 236 bytes from disk
2014-04-10 10:58:02,208 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2014-04-10 10:58:02,208 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2014-04-10 10:58:02,209 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2014-04-10 10:58:02,209 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:02,209 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-10 10:58:02,209 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-10 10:58:02,213 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(test)[by:[{1}:0]]
2014-04-10 10:58:02,213 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simple"]
2014-04-10 10:58:02,217 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local166760767_0015_r_000000_0 is done. And is in the process of committing
2014-04-10 10:58:02,218 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2014-04-10 10:58:02,218 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local166760767_0015_r_000000_0 is allowed to commit now
2014-04-10 10:58:02,219 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local166760767_0015_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simple/_temporary/0/task_local166760767_0015_r_000000
2014-04-10 10:58:02,220 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2014-04-10 10:58:02,220 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local166760767_0015_r_000000_0' done.
2014-04-10 10:58:02,220 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local166760767_0015_r_000000_0
2014-04-10 10:58:02,220 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2014-04-10 10:58:02,234 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output/hadoop2-mr1/basicpipesplatform/simple/_temporary
2014-04-10 10:58:02,249 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2014-04-10 10:58:02,250 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/cloudera-hadoop2-5.0/build/test/output//hadoop2-mr1/basicpipesplatform/simple