2015-02-10 20:22:54,200 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-10 20:22:54,264 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-10 20:22:54,265 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.SortedValuesPlatformTest
2015-02-10 20:22:54,309 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-10 20:22:54,936 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-10 20:22:55,238 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:22:55,240 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:22:55,253 INFO property.AppProps (AppProps.java:getAppID(169)) - using app.id: B6B9DD642AFF4A4FBA0979D6E84D2E09
2015-02-10 20:22:55,345 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:22:55,394 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:22:55,398 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:22:55,536 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-10 20:22:55,539 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting
2015-02-10 20:22:55,540 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt"]
2015-02-10 20:22:55,540 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt"]
2015-02-10 20:22:55,541 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed"]
2015-02-10 20:22:55,542 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] parallel execution is enabled: false
2015-02-10 20:22:55,543 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting jobs: 1
2015-02-10 20:22:55,543 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] allocating threads: 1
2015-02-10 20:22:55,544 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] starting step: (1/1) ...rm/cogroupcomparereversed
2015-02-10 20:22:55,568 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-10 20:22:55,570 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-10 20:22:55,594 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:22:55,973 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:22:55,985 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:22:56,018 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:22:56,188 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local301029041_0001
2015-02-10 20:22:56,231 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity301029041/.staging/job_local301029041_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:22:56,232 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity301029041/.staging/job_local301029041_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:22:56,432 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local301029041_0001/job_local301029041_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:22:56,434 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local301029041_0001/job_local301029041_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:22:56,447 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:22:56,449 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:22:56,452 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] submitted hadoop job: job_local301029041_0001
2015-02-10 20:22:56,453 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] tracking url: http://localhost:8080/
2015-02-10 20:22:56,453 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:22:56,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:22:56,518 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local301029041_0001_m_000000_0
2015-02-10 20:22:56,561 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:22:56,572 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt
2015-02-10 20:22:56,572 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@46e13e7b
2015-02-10 20:22:56,583 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:22:56,591 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:22:56,769 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:22:56,770 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:22:56,770 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:22:56,770 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:22:56,770 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:22:56,795 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:22:56,795 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:22:56,860 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-10 20:22:56,882 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt"]
2015-02-10 20:22:56,883 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:22:56,972 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:22:56,972 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:22:56,972 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:22:56,972 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 9461; bufvoid = 104857600
2015-02-10 20:22:56,973 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2015-02-10 20:23:02,555 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:08,557 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:11,559 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:14,560 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:17,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:19,079 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:19,083 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local301029041_0001_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:19,085 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599
2015-02-10 20:23:19,086 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local301029041_0001_m_000000_0' done.
2015-02-10 20:23:19,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local301029041_0001_m_000000_0
2015-02-10 20:23:19,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local301029041_0001_m_000001_0
2015-02-10 20:23:19,089 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:19,090 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt
2015-02-10 20:23:19,091 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b8a80d4
2015-02-10 20:23:19,092 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:19,093 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:19,254 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:19,254 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:19,254 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:19,254 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:19,255 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:19,257 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:19,257 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:19,278 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt"]
2015-02-10 20:23:19,278 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:23:19,284 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:19,284 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:19,284 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:19,284 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 142; bufvoid = 104857600
2015-02-10 20:23:19,285 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2015-02-10 20:23:20,987 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:20,990 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local301029041_0001_m_000001_0 is done. And is in the process of committing
2015-02-10 20:23:20,993 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt:0+275
2015-02-10 20:23:20,993 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local301029041_0001_m_000001_0' done.
2015-02-10 20:23:20,993 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local301029041_0001_m_000001_0
2015-02-10 20:23:20,994 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:20,996 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:20,997 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local301029041_0001_r_000000_0
2015-02-10 20:23:21,005 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:21,008 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7d16ecf
2015-02-10 20:23:21,024 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:21,027 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local301029041_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:21,053 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local301029041_0001_m_000000_0 decomp: 9863 len: 9867 to MEMORY
2015-02-10 20:23:21,060 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 9863 bytes from map-output for attempt_local301029041_0001_m_000000_0
2015-02-10 20:23:21,097 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 9863, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->9863
2015-02-10 20:23:21,101 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local301029041_0001_m_000001_0 decomp: 184 len: 188 to MEMORY
2015-02-10 20:23:21,102 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 184 bytes from map-output for attempt_local301029041_0001_m_000001_0
2015-02-10 20:23:21,102 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 184, inMemoryMapOutputs.size() -> 2, commitMemory -> 9863, usedMemory ->10047
2015-02-10 20:23:21,103 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:21,105 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:21,105 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:21,114 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:23:21,137 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 10033 bytes
2015-02-10 20:23:25,815 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 10047 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:25,816 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 10049 bytes from disk
2015-02-10 20:23:25,817 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:25,817 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:25,818 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 10038 bytes
2015-02-10 20:23:25,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:25,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:25,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:25,841 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:23:25,841 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed"]
2015-02-10 20:23:25,865 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:23:25,865 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:23:25,920 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local301029041_0001_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:25,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:25,923 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local301029041_0001_r_000000_0 is allowed to commit now
2015-02-10 20:23:25,925 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local301029041_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed/_temporary/0/task_local301029041_0001_r_000000
2015-02-10 20:23:25,927 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:25,927 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local301029041_0001_r_000000_0' done.
2015-02-10 20:23:25,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local301029041_0001_r_000000_0
2015-02-10 20:23:25,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:25,973 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed/_temporary
2015-02-10 20:23:26,015 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:26,055 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/cogroupcomparereversed
2015-02-10 20:23:26,087 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:26,087 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:26,112 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:26,120 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:26,121 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:26,139 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting
2015-02-10 20:23:26,139 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt"]
2015-02-10 20:23:26,139 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt"]
2015-02-10 20:23:26,139 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward"]
2015-02-10 20:23:26,140 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] parallel execution is enabled: false
2015-02-10 20:23:26,140 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] starting jobs: 1
2015-02-10 20:23:26,140 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache*ip] allocating threads: 1
2015-02-10 20:23:26,141 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] starting step: (1/1) ...orm/cogroupcompareforward
2015-02-10 20:23:26,143 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:26,145 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:26,178 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:26,180 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:26,204 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:23:26,241 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local881409907_0002
2015-02-10 20:23:26,259 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity881409907/.staging/job_local881409907_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:26,259 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity881409907/.staging/job_local881409907_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:26,327 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local881409907_0002/job_local881409907_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:26,327 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local881409907_0002/job_local881409907_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:26,330 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:26,332 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] submitted hadoop job: job_local881409907_0002
2015-02-10 20:23:26,332 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache*ip] tracking url: http://localhost:8080/
2015-02-10 20:23:26,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:26,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:26,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:26,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local881409907_0002_m_000000_0
2015-02-10 20:23:26,341 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:26,342 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt
2015-02-10 20:23:26,343 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@788c1852
2015-02-10 20:23:26,344 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:26,344 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:26,362 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:26,362 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:26,362 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:26,363 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:26,363 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:26,365 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:26,365 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:26,383 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt"]
2015-02-10 20:23:26,383 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:23:26,433 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:26,433 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:26,434 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:26,434 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 9461; bufvoid = 104857600
2015-02-10 20:23:26,434 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2015-02-10 20:23:32,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:35,343 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:38,344 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:41,346 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:44,347 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599 > sort
2015-02-10 20:23:44,983 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:44,986 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local881409907_0002_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:44,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/apache.200.txt:0+32599
2015-02-10 20:23:44,990 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local881409907_0002_m_000000_0' done.
2015-02-10 20:23:44,991 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local881409907_0002_m_000000_0
2015-02-10 20:23:44,991 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local881409907_0002_m_000001_0
2015-02-10 20:23:44,993 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:44,995 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt
2015-02-10 20:23:44,995 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3cf27cdc
2015-02-10 20:23:44,999 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:44,999 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:45,022 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:45,022 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:45,023 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:45,023 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:45,023 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:45,025 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:45,025 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:45,041 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt"]
2015-02-10 20:23:45,042 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:23:45,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:45,047 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:45,047 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:45,047 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 142; bufvoid = 104857600
2015-02-10 20:23:45,047 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2015-02-10 20:23:47,014 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:47,043 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local881409907_0002_m_000001_0 is done. And is in the process of committing
2015-02-10 20:23:47,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/ips.20.txt:0+275
2015-02-10 20:23:47,045 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local881409907_0002_m_000001_0' done.
2015-02-10 20:23:47,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local881409907_0002_m_000001_0
2015-02-10 20:23:47,046 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:47,046 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:47,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local881409907_0002_r_000000_0
2015-02-10 20:23:47,049 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:47,049 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6d2d49e7
2015-02-10 20:23:47,051 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:47,052 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local881409907_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:47,055 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local881409907_0002_m_000001_0 decomp: 184 len: 188 to MEMORY
2015-02-10 20:23:47,055 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 184 bytes from map-output for attempt_local881409907_0002_m_000001_0
2015-02-10 20:23:47,056 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 184, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->184
2015-02-10 20:23:47,059 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local881409907_0002_m_000000_0 decomp: 9863 len: 9867 to MEMORY
2015-02-10 20:23:47,060 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 9863 bytes from map-output for attempt_local881409907_0002_m_000000_0
2015-02-10 20:23:47,060 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 9863, inMemoryMapOutputs.size() -> 2, commitMemory -> 184, usedMemory ->10047
2015-02-10 20:23:47,061 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:47,063 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:47,063 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:47,067 INFO mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:23:47,089 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 10035 bytes
2015-02-10 20:23:51,350 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 10047 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:51,351 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 10049 bytes from disk
2015-02-10 20:23:51,352 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:51,353 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:51,353 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 10039 bytes
2015-02-10 20:23:51,355 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:51,356 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:51,357 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:51,372 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(apache*ip)[by: apache:[{1}:'octet'] ip:[{1}:'rawoctet']]
2015-02-10 20:23:51,373 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward"]
2015-02-10 20:23:51,385 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:23:51,386 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:23:51,427 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local881409907_0002_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:51,429 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:23:51,429 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local881409907_0002_r_000000_0 is allowed to commit now
2015-02-10 20:23:51,431 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local881409907_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward/_temporary/0/task_local881409907_0002_r_000000
2015-02-10 20:23:51,433 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:51,433 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local881409907_0002_r_000000_0' done.
2015-02-10 20:23:51,434 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local881409907_0002_r_000000_0
2015-02-10 20:23:51,434 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:51,469 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward/_temporary
2015-02-10 20:23:51,501 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:51,523 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/cogroupcompareforward
2015-02-10 20:23:51,547 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:51,548 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:51,582 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:51,582 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:51,595 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:51,600 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:51,601 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:51,613 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:51,613 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:51,614 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false"]
2015-02-10 20:23:51,614 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:51,614 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:51,614 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:51,615 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_null_false_false
2015-02-10 20:23:51,616 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:51,617 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:51,650 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:51,672 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:51,716 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local489704700_0003
2015-02-10 20:23:51,730 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity489704700/.staging/job_local489704700_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:51,730 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity489704700/.staging/job_local489704700_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:51,783 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local489704700_0003/job_local489704700_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:51,784 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local489704700_0003/job_local489704700_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:51,785 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:51,785 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local489704700_0003
2015-02-10 20:23:51,786 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:51,786 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:51,786 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:51,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:51,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local489704700_0003_m_000000_0
2015-02-10 20:23:51,792 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:51,794 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:51,794 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@420e54f3
2015-02-10 20:23:51,796 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:51,796 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:51,819 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:51,830 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:51,830 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:51,830 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:51,830 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:51,834 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:51,834 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:51,845 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:51,846 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:51,853 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:51,854 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:51,854 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:51,854 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:23:51,854 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:23:51,860 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:51,863 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local489704700_0003_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:51,865 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:23:51,866 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local489704700_0003_m_000000_0' done.
2015-02-10 20:23:51,866 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local489704700_0003_m_000000_0
2015-02-10 20:23:51,867 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:51,867 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:51,868 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local489704700_0003_r_000000_0
2015-02-10 20:23:51,871 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:51,871 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@55a1ad7d
2015-02-10 20:23:51,872 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:51,873 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local489704700_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:51,875 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local489704700_0003_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:23:51,876 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local489704700_0003_m_000000_0
2015-02-10 20:23:51,876 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:23:51,877 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:51,878 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:51,878 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:51,881 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:51,881 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:51,883 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:51,884 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:23:51,884 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:51,884 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:51,885 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:51,885 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:51,886 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:51,886 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:51,896 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:51,896 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false"]
2015-02-10 20:23:51,916 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local489704700_0003_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:51,918 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:51,918 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local489704700_0003_r_000000_0 is allowed to commit now
2015-02-10 20:23:51,919 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local489704700_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false/_temporary/0/task_local489704700_0003_r_000000
2015-02-10 20:23:51,920 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:51,921 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local489704700_0003_r_000000_0' done.
2015-02-10 20:23:51,921 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local489704700_0003_r_000000_0
2015-02-10 20:23:51,921 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:51,954 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false/_temporary
2015-02-10 20:23:51,985 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:52,007 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:52,008 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:52,020 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:52,025 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:52,026 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:52,035 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:52,035 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:52,035 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false"]
2015-02-10 20:23:52,036 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:52,036 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:52,036 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:52,037 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_null_false_false
2015-02-10 20:23:52,038 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:52,039 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:52,071 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:52,096 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:52,114 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1094697481_0004
2015-02-10 20:23:52,128 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1094697481/.staging/job_local1094697481_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:52,128 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1094697481/.staging/job_local1094697481_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:52,172 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1094697481_0004/job_local1094697481_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:52,173 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1094697481_0004/job_local1094697481_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:52,174 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:52,177 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1094697481_0004
2015-02-10 20:23:52,177 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:52,177 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:52,178 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:52,182 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:52,182 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1094697481_0004_m_000000_0
2015-02-10 20:23:52,186 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:52,187 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:52,188 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@306e7a5b
2015-02-10 20:23:52,190 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:52,190 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:52,216 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:52,216 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:52,216 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:52,217 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:52,217 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:52,223 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:52,223 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:52,234 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:52,234 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:52,241 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:52,241 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:52,241 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:52,242 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:23:52,242 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:23:54,636 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:54,638 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1094697481_0004_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:54,641 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:23:54,641 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1094697481_0004_m_000000_0' done.
2015-02-10 20:23:54,641 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1094697481_0004_m_000000_0
2015-02-10 20:23:54,641 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:54,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:54,642 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1094697481_0004_r_000000_0
2015-02-10 20:23:54,645 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:54,645 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@dee55c
2015-02-10 20:23:54,646 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:54,647 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1094697481_0004_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:54,649 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local1094697481_0004_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:23:54,650 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1094697481_0004_m_000000_0
2015-02-10 20:23:54,651 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:23:54,651 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:54,653 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:54,653 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:54,656 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:54,656 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:54,658 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:54,659 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:23:54,659 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:54,659 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:54,659 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:54,660 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:54,661 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:54,661 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:54,670 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:54,670 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false"]
2015-02-10 20:23:54,687 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1094697481_0004_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:54,689 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:54,690 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1094697481_0004_r_000000_0 is allowed to commit now
2015-02-10 20:23:54,691 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1094697481_0004_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false/_temporary/0/task_local1094697481_0004_r_000000
2015-02-10 20:23:54,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:54,694 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1094697481_0004_r_000000_0' done.
2015-02-10 20:23:54,694 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1094697481_0004_r_000000_0
2015-02-10 20:23:54,695 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:54,724 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false/_temporary
2015-02-10 20:23:54,753 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:54,780 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:54,781 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:54,793 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:54,798 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:54,799 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:54,809 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:54,809 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:54,810 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false"]
2015-02-10 20:23:54,810 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:54,811 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:54,811 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:54,811 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_true_false
2015-02-10 20:23:54,813 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:54,814 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:54,842 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:54,862 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:54,895 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1909296994_0005
2015-02-10 20:23:54,908 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1909296994/.staging/job_local1909296994_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:54,909 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1909296994/.staging/job_local1909296994_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:54,953 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1909296994_0005/job_local1909296994_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:54,953 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1909296994_0005/job_local1909296994_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:54,954 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:54,955 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1909296994_0005
2015-02-10 20:23:54,955 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:54,955 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:54,955 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:54,960 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:54,961 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1909296994_0005_m_000000_0
2015-02-10 20:23:54,963 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:54,964 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:54,964 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48a70acd
2015-02-10 20:23:54,966 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:54,966 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:54,983 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:54,984 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:54,984 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:54,984 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:54,984 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:54,987 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:54,988 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:55,001 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:55,001 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:55,008 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:55,009 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:55,009 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:55,009 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:23:55,009 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:23:55,013 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:55,016 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1909296994_0005_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:55,018 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:23:55,018 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1909296994_0005_m_000000_0' done.
2015-02-10 20:23:55,018 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1909296994_0005_m_000000_0
2015-02-10 20:23:55,019 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:55,019 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:55,020 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1909296994_0005_r_000000_0
2015-02-10 20:23:55,022 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:55,022 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@d5c4ee6
2015-02-10 20:23:55,023 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:55,023 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1909296994_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:55,025 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local1909296994_0005_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:23:55,026 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1909296994_0005_m_000000_0
2015-02-10 20:23:55,026 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:23:55,027 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:55,028 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:55,028 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:55,032 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:55,032 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:55,034 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:55,034 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:23:55,035 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:55,035 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:55,035 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:55,036 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:55,036 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:55,036 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:55,045 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:55,045 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false"]
2015-02-10 20:23:55,061 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1909296994_0005_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:55,062 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:55,062 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1909296994_0005_r_000000_0 is allowed to commit now
2015-02-10 20:23:55,063 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1909296994_0005_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false/_temporary/0/task_local1909296994_0005_r_000000
2015-02-10 20:23:55,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:55,065 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1909296994_0005_r_000000_0' done.
2015-02-10 20:23:55,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1909296994_0005_r_000000_0
2015-02-10 20:23:55,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:55,086 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false/_temporary
2015-02-10 20:23:55,114 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:55,134 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:55,136 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:55,147 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:55,151 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:55,152 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:55,161 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:55,161 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:55,162 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false"]
2015-02-10 20:23:55,162 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:55,163 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:55,163 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:55,164 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_true_false
2015-02-10 20:23:55,165 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:55,166 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:55,194 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:55,213 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:55,230 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1185410763_0006
2015-02-10 20:23:55,243 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1185410763/.staging/job_local1185410763_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:55,244 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1185410763/.staging/job_local1185410763_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:55,287 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1185410763_0006/job_local1185410763_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:55,287 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1185410763_0006/job_local1185410763_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:55,289 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:55,289 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1185410763_0006
2015-02-10 20:23:55,289 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:55,289 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:55,289 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:55,293 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:55,293 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1185410763_0006_m_000000_0
2015-02-10 20:23:55,295 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:55,297 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:55,297 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@57202a06
2015-02-10 20:23:55,299 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:55,299 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:55,319 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:55,320 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:55,320 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:55,320 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:55,320 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:55,323 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:55,323 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:55,335 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:55,335 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:55,342 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:55,342 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:55,342 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:55,342 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:23:55,343 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:23:57,763 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:57,766 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1185410763_0006_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:57,768 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:23:57,768 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1185410763_0006_m_000000_0' done.
2015-02-10 20:23:57,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1185410763_0006_m_000000_0
2015-02-10 20:23:57,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:57,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:57,770 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1185410763_0006_r_000000_0
2015-02-10 20:23:57,772 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:57,772 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@38c9aa93
2015-02-10 20:23:57,773 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:57,774 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1185410763_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:57,775 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local1185410763_0006_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:23:57,776 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1185410763_0006_m_000000_0
2015-02-10 20:23:57,776 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:23:57,777 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:57,778 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:57,779 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:57,783 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:57,783 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:57,785 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:57,786 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:23:57,787 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:57,787 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:57,788 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:23:57,789 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:57,790 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:57,791 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:57,800 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:57,800 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false"]
2015-02-10 20:23:57,818 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1185410763_0006_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:57,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:57,820 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1185410763_0006_r_000000_0 is allowed to commit now
2015-02-10 20:23:57,821 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1185410763_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false/_temporary/0/task_local1185410763_0006_r_000000
2015-02-10 20:23:57,823 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:57,823 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1185410763_0006_r_000000_0' done.
2015-02-10 20:23:57,823 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1185410763_0006_r_000000_0
2015-02-10 20:23:57,824 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:57,848 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false/_temporary
2015-02-10 20:23:57,876 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:57,903 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:57,903 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:57,913 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:57,917 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:57,918 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:57,928 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:57,928 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:57,928 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false"]
2015-02-10 20:23:57,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:57,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:57,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:57,930 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...e/false_false_false_false
2015-02-10 20:23:57,931 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:57,932 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:57,958 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:57,980 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:57,997 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local911090351_0007
2015-02-10 20:23:58,011 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity911090351/.staging/job_local911090351_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:58,011 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity911090351/.staging/job_local911090351_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:58,053 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local911090351_0007/job_local911090351_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:58,054 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local911090351_0007/job_local911090351_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:58,055 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:58,055 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local911090351_0007
2015-02-10 20:23:58,055 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:58,056 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:58,055 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:58,059 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:58,059 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local911090351_0007_m_000000_0
2015-02-10 20:23:58,061 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:58,062 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:58,063 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4785477d
2015-02-10 20:23:58,064 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:58,065 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:58,082 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:58,082 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:58,082 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:58,082 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:58,083 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:58,085 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:58,085 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:58,094 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:58,095 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:58,102 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:58,102 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:58,102 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:58,103 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:23:58,103 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:23:58,108 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:23:58,110 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local911090351_0007_m_000000_0 is done. And is in the process of committing
2015-02-10 20:23:58,112 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:23:58,113 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local911090351_0007_m_000000_0' done.
2015-02-10 20:23:58,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local911090351_0007_m_000000_0
2015-02-10 20:23:58,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:23:58,114 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:23:58,114 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local911090351_0007_r_000000_0
2015-02-10 20:23:58,116 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:58,116 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@500b3f4c
2015-02-10 20:23:58,117 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:23:58,118 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local911090351_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:23:58,120 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local911090351_0007_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:23:58,120 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local911090351_0007_m_000000_0
2015-02-10 20:23:58,120 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:23:58,121 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:23:58,122 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:58,122 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:23:58,125 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:58,126 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:23:58,128 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:23:58,128 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:23:58,128 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:23:58,129 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:23:58,129 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:23:58,130 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:58,130 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:23:58,131 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:23:58,139 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:58,139 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false"]
2015-02-10 20:23:58,156 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local911090351_0007_r_000000_0 is done. And is in the process of committing
2015-02-10 20:23:58,157 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:23:58,158 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local911090351_0007_r_000000_0 is allowed to commit now
2015-02-10 20:23:58,158 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local911090351_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false/_temporary/0/task_local911090351_0007_r_000000
2015-02-10 20:23:58,160 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:23:58,160 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local911090351_0007_r_000000_0' done.
2015-02-10 20:23:58,160 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local911090351_0007_r_000000_0
2015-02-10 20:23:58,161 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:23:58,185 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false/_temporary
2015-02-10 20:23:58,213 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:58,232 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:23:58,233 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:23:58,243 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:23:58,247 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:23:58,248 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:23:58,256 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:23:58,256 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:58,256 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false"]
2015-02-10 20:23:58,256 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:23:58,257 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:23:58,257 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:23:58,258 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/true_false_false_false
2015-02-10 20:23:58,259 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:58,260 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:23:58,287 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:23:58,307 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:23:58,323 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local405703034_0008
2015-02-10 20:23:58,337 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity405703034/.staging/job_local405703034_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:58,337 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity405703034/.staging/job_local405703034_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:58,379 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local405703034_0008/job_local405703034_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:23:58,379 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local405703034_0008/job_local405703034_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:23:58,380 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:23:58,380 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local405703034_0008
2015-02-10 20:23:58,380 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:23:58,381 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:23:58,381 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:23:58,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:23:58,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local405703034_0008_m_000000_0
2015-02-10 20:23:58,386 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:23:58,388 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:23:58,388 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@e62ce0a
2015-02-10 20:23:58,389 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:23:58,389 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:23:58,412 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:23:58,413 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:23:58,413 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:23:58,414 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:23:58,414 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:23:58,416 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:23:58,416 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:23:58,425 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:23:58,425 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:23:58,433 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:23:58,433 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:23:58,433 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:23:58,433 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:23:58,433 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:01,890 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:01,893 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local405703034_0008_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:01,895 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:01,895 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local405703034_0008_m_000000_0' done.
2015-02-10 20:24:01,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local405703034_0008_m_000000_0
2015-02-10 20:24:01,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:01,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:01,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local405703034_0008_r_000000_0
2015-02-10 20:24:01,898 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:01,899 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5e1f5fc7
2015-02-10 20:24:01,900 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:01,901 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local405703034_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:01,903 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local405703034_0008_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:01,904 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local405703034_0008_m_000000_0
2015-02-10 20:24:01,904 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:01,906 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:01,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:01,908 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:01,911 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:01,911 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:01,914 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:01,915 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:01,915 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:01,915 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:01,916 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:01,917 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:01,918 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:01,918 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:01,927 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:01,928 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false"]
2015-02-10 20:24:01,944 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local405703034_0008_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:01,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:01,946 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local405703034_0008_r_000000_0 is allowed to commit now
2015-02-10 20:24:01,947 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local405703034_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false/_temporary/0/task_local405703034_0008_r_000000
2015-02-10 20:24:01,948 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:01,949 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local405703034_0008_r_000000_0' done.
2015-02-10 20:24:01,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local405703034_0008_r_000000_0
2015-02-10 20:24:01,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:01,969 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false/_temporary
2015-02-10 20:24:01,996 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:02,016 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:02,017 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:02,027 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:02,029 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:02,030 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:02,040 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:02,040 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:02,041 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false"]
2015-02-10 20:24:02,041 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:02,041 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:02,041 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:02,042 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_true_false_false
2015-02-10 20:24:02,042 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:02,043 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:02,069 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:02,090 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:02,112 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local855145230_0009
2015-02-10 20:24:02,127 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity855145230/.staging/job_local855145230_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:02,127 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity855145230/.staging/job_local855145230_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:02,169 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local855145230_0009/job_local855145230_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:02,169 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local855145230_0009/job_local855145230_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:02,170 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:02,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:02,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:02,171 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local855145230_0009
2015-02-10 20:24:02,172 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:02,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:02,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local855145230_0009_m_000000_0
2015-02-10 20:24:02,176 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:02,178 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:02,178 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@149be3aa
2015-02-10 20:24:02,179 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:02,180 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:02,197 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:02,197 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:02,197 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:02,198 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:02,198 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:02,200 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:02,200 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:02,209 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:02,209 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:02,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:02,216 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:02,216 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:02,216 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:02,217 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:06,607 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:06,609 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local855145230_0009_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:06,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:06,612 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local855145230_0009_m_000000_0' done.
2015-02-10 20:24:06,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local855145230_0009_m_000000_0
2015-02-10 20:24:06,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:06,613 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:06,613 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local855145230_0009_r_000000_0
2015-02-10 20:24:06,615 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:06,615 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6c20a239
2015-02-10 20:24:06,616 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:06,617 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local855145230_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:06,619 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local855145230_0009_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:06,620 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local855145230_0009_m_000000_0
2015-02-10 20:24:06,620 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:06,622 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:06,623 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:06,623 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:06,626 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:06,626 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:06,628 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:06,629 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:06,629 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:06,629 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:06,630 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:06,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:06,631 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:06,631 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:06,639 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:06,639 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false"]
2015-02-10 20:24:06,655 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local855145230_0009_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:06,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:06,657 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local855145230_0009_r_000000_0 is allowed to commit now
2015-02-10 20:24:06,658 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local855145230_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false/_temporary/0/task_local855145230_0009_r_000000
2015-02-10 20:24:06,659 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:06,659 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local855145230_0009_r_000000_0' done.
2015-02-10 20:24:06,659 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local855145230_0009_r_000000_0
2015-02-10 20:24:06,659 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:06,680 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false/_temporary
2015-02-10 20:24:06,708 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:06,727 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:06,728 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:06,737 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:06,739 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:06,740 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:06,747 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:06,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:06,748 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false"]
2015-02-10 20:24:06,749 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:06,749 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:06,749 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:06,750 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_true_false_false
2015-02-10 20:24:06,751 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:06,752 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:06,778 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:06,798 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:06,814 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local283911924_0010
2015-02-10 20:24:06,828 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity283911924/.staging/job_local283911924_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:06,828 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity283911924/.staging/job_local283911924_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:06,870 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local283911924_0010/job_local283911924_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:06,870 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local283911924_0010/job_local283911924_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:06,871 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:06,871 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local283911924_0010
2015-02-10 20:24:06,871 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:06,872 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:06,872 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:06,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:06,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local283911924_0010_m_000000_0
2015-02-10 20:24:06,877 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:06,879 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:06,879 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2a51b326
2015-02-10 20:24:06,880 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:06,881 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:06,904 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:06,904 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:06,904 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:06,904 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:06,904 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:06,906 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:06,906 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:06,917 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:06,917 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:06,923 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:06,924 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:06,924 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:06,924 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:06,924 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:09,238 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:09,241 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local283911924_0010_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:09,243 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:09,243 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local283911924_0010_m_000000_0' done.
2015-02-10 20:24:09,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local283911924_0010_m_000000_0
2015-02-10 20:24:09,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:09,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:09,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local283911924_0010_r_000000_0
2015-02-10 20:24:09,247 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:09,247 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33d8053c
2015-02-10 20:24:09,249 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:09,250 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local283911924_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:09,252 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local283911924_0010_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:09,253 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local283911924_0010_m_000000_0
2015-02-10 20:24:09,253 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:09,254 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:09,256 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,256 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:09,259 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:09,259 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:09,261 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:09,261 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:09,261 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:09,261 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:09,262 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:09,262 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,263 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:09,263 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:09,271 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:09,271 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false"]
2015-02-10 20:24:09,286 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local283911924_0010_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:09,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,288 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local283911924_0010_r_000000_0 is allowed to commit now
2015-02-10 20:24:09,289 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local283911924_0010_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false/_temporary/0/task_local283911924_0010_r_000000
2015-02-10 20:24:09,291 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:09,291 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local283911924_0010_r_000000_0' done.
2015-02-10 20:24:09,291 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local283911924_0010_r_000000_0
2015-02-10 20:24:09,292 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:09,312 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false/_temporary
2015-02-10 20:24:09,340 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:09,360 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:09,361 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:09,370 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:09,372 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:09,373 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:09,380 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:09,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:09,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false"]
2015-02-10 20:24:09,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:09,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:09,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:09,382 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_true_false
2015-02-10 20:24:09,383 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:09,384 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:09,409 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:09,430 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:09,446 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local217320390_0011
2015-02-10 20:24:09,461 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity217320390/.staging/job_local217320390_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:09,461 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity217320390/.staging/job_local217320390_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:09,501 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local217320390_0011/job_local217320390_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:09,501 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local217320390_0011/job_local217320390_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:09,502 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:09,503 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local217320390_0011
2015-02-10 20:24:09,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:09,503 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:09,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:09,506 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:09,506 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local217320390_0011_m_000000_0
2015-02-10 20:24:09,508 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:09,510 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:09,510 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5178efd5
2015-02-10 20:24:09,511 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:09,511 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:09,529 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:09,530 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:09,530 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:09,530 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:09,530 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:09,533 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:09,533 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:09,543 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:09,544 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:09,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:09,551 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:09,551 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:09,551 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:09,551 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:09,556 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:09,558 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local217320390_0011_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:09,560 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:09,560 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local217320390_0011_m_000000_0' done.
2015-02-10 20:24:09,561 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local217320390_0011_m_000000_0
2015-02-10 20:24:09,561 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:09,561 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:09,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local217320390_0011_r_000000_0
2015-02-10 20:24:09,563 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:09,564 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@39563a3d
2015-02-10 20:24:09,564 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:09,565 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local217320390_0011_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:09,567 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local217320390_0011_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:09,567 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local217320390_0011_m_000000_0
2015-02-10 20:24:09,568 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:09,568 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:09,569 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,569 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:09,571 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:09,572 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:09,574 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:09,574 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:09,574 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:09,574 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:09,575 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:09,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,576 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:09,576 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:09,584 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:09,584 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false"]
2015-02-10 20:24:09,600 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local217320390_0011_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:09,601 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:09,601 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local217320390_0011_r_000000_0 is allowed to commit now
2015-02-10 20:24:09,602 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local217320390_0011_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false/_temporary/0/task_local217320390_0011_r_000000
2015-02-10 20:24:09,604 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:09,604 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local217320390_0011_r_000000_0' done.
2015-02-10 20:24:09,604 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local217320390_0011_r_000000_0
2015-02-10 20:24:09,604 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:09,622 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false/_temporary
2015-02-10 20:24:09,649 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:09,669 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:09,670 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:09,678 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:09,680 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:09,681 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false"]
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:09,688 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:09,689 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_true_false
2015-02-10 20:24:09,690 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:09,691 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:09,715 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:09,734 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:09,749 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local475906886_0012
2015-02-10 20:24:09,763 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity475906886/.staging/job_local475906886_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:09,764 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity475906886/.staging/job_local475906886_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:09,806 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local475906886_0012/job_local475906886_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:09,806 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local475906886_0012/job_local475906886_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:09,808 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:09,808 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local475906886_0012
2015-02-10 20:24:09,808 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:09,809 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:09,809 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:09,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:09,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local475906886_0012_m_000000_0
2015-02-10 20:24:09,814 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:09,815 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:09,815 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4a8270d7
2015-02-10 20:24:09,816 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:09,817 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:09,837 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:09,837 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:09,837 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:09,837 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:09,838 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:09,839 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:09,840 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:09,848 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:09,848 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:09,855 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:09,855 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:09,855 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:09,855 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:09,855 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:13,336 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:13,338 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local475906886_0012_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:13,340 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:13,340 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local475906886_0012_m_000000_0' done.
2015-02-10 20:24:13,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local475906886_0012_m_000000_0
2015-02-10 20:24:13,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:13,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:13,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local475906886_0012_r_000000_0
2015-02-10 20:24:13,343 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:13,344 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@69e295f8
2015-02-10 20:24:13,344 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:13,346 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local475906886_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:13,348 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local475906886_0012_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:13,349 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local475906886_0012_m_000000_0
2015-02-10 20:24:13,349 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:13,350 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:13,351 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:13,351 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:13,355 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:13,355 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:13,357 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:13,357 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:13,357 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:13,357 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:13,358 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:13,358 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:13,359 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:13,359 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:13,367 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:13,367 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false"]
2015-02-10 20:24:13,382 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local475906886_0012_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:13,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:13,384 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local475906886_0012_r_000000_0 is allowed to commit now
2015-02-10 20:24:13,385 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local475906886_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false/_temporary/0/task_local475906886_0012_r_000000
2015-02-10 20:24:13,386 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:13,387 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local475906886_0012_r_000000_0' done.
2015-02-10 20:24:13,387 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local475906886_0012_r_000000_0
2015-02-10 20:24:13,387 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:13,406 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false/_temporary
2015-02-10 20:24:13,433 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:13,453 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:13,454 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:13,462 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:13,465 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:13,466 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:13,472 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:13,473 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:13,473 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false"]
2015-02-10 20:24:13,473 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:13,473 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:13,473 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:13,474 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_true_false
2015-02-10 20:24:13,476 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:13,477 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:13,503 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:13,523 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:13,539 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local576590979_0013
2015-02-10 20:24:13,553 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity576590979/.staging/job_local576590979_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:13,553 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity576590979/.staging/job_local576590979_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:13,600 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local576590979_0013/job_local576590979_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:13,601 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local576590979_0013/job_local576590979_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:13,602 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:13,602 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local576590979_0013
2015-02-10 20:24:13,602 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:13,602 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:13,603 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:13,606 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:13,606 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local576590979_0013_m_000000_0
2015-02-10 20:24:13,609 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:13,611 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:13,611 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2fecdb1b
2015-02-10 20:24:13,612 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:13,612 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:13,630 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:13,630 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:13,630 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:13,630 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:13,631 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:13,633 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:13,633 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:13,641 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:13,641 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:13,648 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:13,648 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:13,648 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:13,648 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:13,648 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:18,006 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:18,008 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local576590979_0013_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:18,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:18,011 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local576590979_0013_m_000000_0' done.
2015-02-10 20:24:18,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local576590979_0013_m_000000_0
2015-02-10 20:24:18,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:18,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:18,012 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local576590979_0013_r_000000_0
2015-02-10 20:24:18,014 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:18,014 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1815a349
2015-02-10 20:24:18,015 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:18,017 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local576590979_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:18,018 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local576590979_0013_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:18,019 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local576590979_0013_m_000000_0
2015-02-10 20:24:18,019 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:18,020 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:18,021 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:18,022 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:18,025 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:18,026 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:18,028 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:18,028 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:18,029 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:18,029 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:18,030 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:18,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:18,032 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:18,032 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:18,040 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:18,040 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false"]
2015-02-10 20:24:18,056 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local576590979_0013_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:18,058 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:18,058 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local576590979_0013_r_000000_0 is allowed to commit now
2015-02-10 20:24:18,059 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local576590979_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false/_temporary/0/task_local576590979_0013_r_000000
2015-02-10 20:24:18,060 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:18,060 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local576590979_0013_r_000000_0' done.
2015-02-10 20:24:18,060 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local576590979_0013_r_000000_0
2015-02-10 20:24:18,060 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:18,085 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false/_temporary
2015-02-10 20:24:18,111 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:18,129 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:18,129 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:18,137 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:18,138 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:18,139 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:18,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:18,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:18,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false"]
2015-02-10 20:24:18,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:18,147 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:18,147 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:18,148 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_true_false
2015-02-10 20:24:18,148 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:18,149 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:18,174 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:18,193 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:18,208 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1891791163_0014
2015-02-10 20:24:18,221 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1891791163/.staging/job_local1891791163_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:18,221 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1891791163/.staging/job_local1891791163_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:18,261 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1891791163_0014/job_local1891791163_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:18,261 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1891791163_0014/job_local1891791163_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:18,262 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:18,262 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1891791163_0014
2015-02-10 20:24:18,263 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:18,263 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:18,263 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:18,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:18,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1891791163_0014_m_000000_0
2015-02-10 20:24:18,268 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:18,270 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:18,270 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@a93370a
2015-02-10 20:24:18,271 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:18,272 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:18,295 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:18,295 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:18,296 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:18,296 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:18,296 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:18,298 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:18,298 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:18,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:18,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:18,313 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:18,313 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:18,313 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:18,313 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:18,313 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:20,543 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:20,545 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1891791163_0014_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:20,547 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:20,547 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1891791163_0014_m_000000_0' done.
2015-02-10 20:24:20,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1891791163_0014_m_000000_0
2015-02-10 20:24:20,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:20,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:20,549 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1891791163_0014_r_000000_0
2015-02-10 20:24:20,551 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:20,551 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@432c4c7a
2015-02-10 20:24:20,552 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:20,553 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1891791163_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:20,554 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1891791163_0014_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:20,555 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1891791163_0014_m_000000_0
2015-02-10 20:24:20,555 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:20,556 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:20,557 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,557 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:20,560 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:20,560 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:20,562 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:20,563 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:20,563 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:20,563 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:20,564 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:20,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,565 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:20,565 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:20,573 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:20,574 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false"]
2015-02-10 20:24:20,589 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1891791163_0014_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:20,590 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,590 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1891791163_0014_r_000000_0 is allowed to commit now
2015-02-10 20:24:20,591 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1891791163_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false/_temporary/0/task_local1891791163_0014_r_000000
2015-02-10 20:24:20,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:20,593 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1891791163_0014_r_000000_0' done.
2015-02-10 20:24:20,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1891791163_0014_r_000000_0
2015-02-10 20:24:20,593 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:20,612 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false/_temporary
2015-02-10 20:24:20,639 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:20,657 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:20,657 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:20,665 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:20,667 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:20,668 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:20,673 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true"]
2015-02-10 20:24:20,674 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:20,674 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:20,674 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:20,675 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_null_false_true
2015-02-10 20:24:20,676 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:20,677 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:20,701 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:20,724 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:20,739 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1384541629_0015
2015-02-10 20:24:20,753 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1384541629/.staging/job_local1384541629_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:20,754 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1384541629/.staging/job_local1384541629_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:20,793 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1384541629_0015/job_local1384541629_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:20,793 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1384541629_0015/job_local1384541629_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:20,794 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:20,795 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1384541629_0015
2015-02-10 20:24:20,795 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:20,795 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:20,796 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:20,798 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:20,798 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1384541629_0015_m_000000_0
2015-02-10 20:24:20,800 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:20,801 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:20,802 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3c22de3d
2015-02-10 20:24:20,803 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:20,803 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:20,820 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:20,821 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:20,821 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:20,821 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:20,821 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:20,822 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:20,822 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:20,830 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:20,830 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:20,838 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:20,838 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:20,838 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:20,838 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:24:20,838 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:20,842 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:20,843 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1384541629_0015_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:20,845 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:20,845 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1384541629_0015_m_000000_0' done.
2015-02-10 20:24:20,846 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1384541629_0015_m_000000_0
2015-02-10 20:24:20,846 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:20,846 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:20,847 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1384541629_0015_r_000000_0
2015-02-10 20:24:20,848 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:20,848 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@59c33aa
2015-02-10 20:24:20,849 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:20,850 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1384541629_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:20,851 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1384541629_0015_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:24:20,852 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local1384541629_0015_m_000000_0
2015-02-10 20:24:20,852 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:24:20,853 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:20,854 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,854 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:20,856 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:20,856 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:20,859 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:20,859 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:24:20,860 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:20,860 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:20,860 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:20,861 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,861 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:20,862 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:20,869 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:20,869 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true"]
2015-02-10 20:24:20,883 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1384541629_0015_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:20,884 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:20,885 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1384541629_0015_r_000000_0 is allowed to commit now
2015-02-10 20:24:20,885 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1384541629_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true/_temporary/0/task_local1384541629_0015_r_000000
2015-02-10 20:24:20,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:20,887 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1384541629_0015_r_000000_0' done.
2015-02-10 20:24:20,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1384541629_0015_r_000000_0
2015-02-10 20:24:20,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:20,903 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true/_temporary
2015-02-10 20:24:20,929 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:20,947 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:20,948 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:20,955 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:20,958 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:20,959 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:20,967 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:20,967 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:20,968 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true"]
2015-02-10 20:24:20,968 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:20,968 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:20,969 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:20,970 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_null_false_true
2015-02-10 20:24:20,971 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:20,972 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:20,998 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,016 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:21,030 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local383689037_0016
2015-02-10 20:24:21,044 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity383689037/.staging/job_local383689037_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,044 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity383689037/.staging/job_local383689037_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,084 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local383689037_0016/job_local383689037_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,084 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local383689037_0016/job_local383689037_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,085 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:21,086 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local383689037_0016
2015-02-10 20:24:21,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:21,086 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:21,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:21,089 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:21,089 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local383689037_0016_m_000000_0
2015-02-10 20:24:21,090 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,092 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:21,092 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@41f4fa12
2015-02-10 20:24:21,093 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:21,093 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:21,116 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:21,116 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:21,116 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:21,117 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:21,117 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:21,118 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:21,119 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,127 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,127 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,133 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:21,133 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:21,134 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:21,134 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:24:21,134 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:21,137 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:21,140 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local383689037_0016_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,142 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:21,142 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local383689037_0016_m_000000_0' done.
2015-02-10 20:24:21,142 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local383689037_0016_m_000000_0
2015-02-10 20:24:21,143 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:21,143 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:21,143 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local383689037_0016_r_000000_0
2015-02-10 20:24:21,145 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,146 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@299966f0
2015-02-10 20:24:21,147 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:21,148 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local383689037_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:21,150 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local383689037_0016_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:24:21,150 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local383689037_0016_m_000000_0
2015-02-10 20:24:21,150 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:24:21,151 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:21,152 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,152 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:21,155 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,155 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,157 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:21,158 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:24:21,158 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:21,158 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,159 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,160 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,161 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:21,162 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,169 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,169 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true"]
2015-02-10 20:24:21,183 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local383689037_0016_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,185 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,185 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local383689037_0016_r_000000_0 is allowed to commit now
2015-02-10 20:24:21,186 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local383689037_0016_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true/_temporary/0/task_local383689037_0016_r_000000
2015-02-10 20:24:21,187 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:21,187 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local383689037_0016_r_000000_0' done.
2015-02-10 20:24:21,187 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local383689037_0016_r_000000_0
2015-02-10 20:24:21,187 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:21,204 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true/_temporary
2015-02-10 20:24:21,231 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,251 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:21,251 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:21,259 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:21,262 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:21,262 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true"]
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:21,268 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:21,269 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/false_null_true_true
2015-02-10 20:24:21,270 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,271 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,295 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,314 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:21,328 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local914644300_0017
2015-02-10 20:24:21,342 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity914644300/.staging/job_local914644300_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,342 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity914644300/.staging/job_local914644300_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,381 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local914644300_0017/job_local914644300_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,382 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local914644300_0017/job_local914644300_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,384 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:21,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:21,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:21,384 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local914644300_0017
2015-02-10 20:24:21,385 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:21,387 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:21,387 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local914644300_0017_m_000000_0
2015-02-10 20:24:21,389 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,390 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:21,390 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6cd97a79
2015-02-10 20:24:21,391 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:21,391 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:21,414 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:21,414 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:21,414 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:21,415 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:21,415 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:21,416 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:21,416 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,424 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,424 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,430 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:21,430 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:21,430 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:21,430 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:24:21,430 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:21,433 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:21,435 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local914644300_0017_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,437 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:21,437 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local914644300_0017_m_000000_0' done.
2015-02-10 20:24:21,437 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local914644300_0017_m_000000_0
2015-02-10 20:24:21,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:21,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:21,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local914644300_0017_r_000000_0
2015-02-10 20:24:21,440 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,440 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@132b1b6c
2015-02-10 20:24:21,441 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:21,442 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local914644300_0017_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:21,445 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local914644300_0017_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:24:21,445 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local914644300_0017_m_000000_0
2015-02-10 20:24:21,446 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:24:21,446 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:21,447 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,447 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:21,452 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,452 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,454 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:21,454 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:24:21,455 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:21,455 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,455 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,456 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:21,456 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,463 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,464 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true"]
2015-02-10 20:24:21,477 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local914644300_0017_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,479 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,479 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local914644300_0017_r_000000_0 is allowed to commit now
2015-02-10 20:24:21,480 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local914644300_0017_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true/_temporary/0/task_local914644300_0017_r_000000
2015-02-10 20:24:21,481 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:21,482 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local914644300_0017_r_000000_0' done.
2015-02-10 20:24:21,482 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local914644300_0017_r_000000_0
2015-02-10 20:24:21,482 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:21,501 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true/_temporary
2015-02-10 20:24:21,527 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,546 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:21,546 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:21,554 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:21,556 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:21,556 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:21,561 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:21,561 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,561 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true"]
2015-02-10 20:24:21,562 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:21,562 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:21,562 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:21,562 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...nsive/true_null_true_true
2015-02-10 20:24:21,563 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,564 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,588 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,606 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:21,622 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local540279939_0018
2015-02-10 20:24:21,637 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity540279939/.staging/job_local540279939_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,637 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity540279939/.staging/job_local540279939_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,679 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local540279939_0018/job_local540279939_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,680 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local540279939_0018/job_local540279939_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,681 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:21,682 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:21,682 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:21,682 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local540279939_0018
2015-02-10 20:24:21,683 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:21,685 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:21,685 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local540279939_0018_m_000000_0
2015-02-10 20:24:21,686 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,688 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:21,688 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@70413ea6
2015-02-10 20:24:21,689 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:21,689 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:21,712 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:21,712 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:21,712 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:21,712 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:21,713 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:21,714 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:21,714 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,722 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,722 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,728 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:21,728 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:21,728 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:21,728 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 629; bufvoid = 104857600
2015-02-10 20:24:21,728 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:21,732 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:21,733 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local540279939_0018_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:21,735 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local540279939_0018_m_000000_0' done.
2015-02-10 20:24:21,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local540279939_0018_m_000000_0
2015-02-10 20:24:21,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:21,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:21,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local540279939_0018_r_000000_0
2015-02-10 20:24:21,738 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,738 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6ee00ed0
2015-02-10 20:24:21,739 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:21,740 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local540279939_0018_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:21,741 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local540279939_0018_m_000000_0 decomp: 705 len: 709 to MEMORY
2015-02-10 20:24:21,742 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 705 bytes from map-output for attempt_local540279939_0018_m_000000_0
2015-02-10 20:24:21,742 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 705, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->705
2015-02-10 20:24:21,744 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:21,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,745 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:21,748 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,748 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,749 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 705 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:21,750 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 709 bytes from disk
2015-02-10 20:24:21,750 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:21,750 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:21,750 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 700 bytes
2015-02-10 20:24:21,751 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,752 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:21,752 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:21,759 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:21,759 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true"]
2015-02-10 20:24:21,773 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local540279939_0018_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:21,774 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:21,774 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local540279939_0018_r_000000_0 is allowed to commit now
2015-02-10 20:24:21,775 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local540279939_0018_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true/_temporary/0/task_local540279939_0018_r_000000
2015-02-10 20:24:21,776 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:21,776 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local540279939_0018_r_000000_0' done.
2015-02-10 20:24:21,777 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local540279939_0018_r_000000_0
2015-02-10 20:24:21,777 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:21,799 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true/_temporary
2015-02-10 20:24:21,826 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,843 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:21,843 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:21,851 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:21,852 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:21,853 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true"]
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:21,858 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:21,859 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ve/false_false_false_true
2015-02-10 20:24:21,860 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,861 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:21,884 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:21,902 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:21,917 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local216527774_0019
2015-02-10 20:24:21,932 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity216527774/.staging/job_local216527774_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,932 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity216527774/.staging/job_local216527774_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,973 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local216527774_0019/job_local216527774_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:21,973 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local216527774_0019/job_local216527774_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:21,974 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:21,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:21,975 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local216527774_0019
2015-02-10 20:24:21,975 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:21,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:21,978 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:21,978 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local216527774_0019_m_000000_0
2015-02-10 20:24:21,980 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:21,981 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:21,981 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5537c1f1
2015-02-10 20:24:21,982 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:21,982 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:22,006 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:22,006 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:22,006 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:22,006 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:22,006 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:22,007 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:22,007 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,015 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,015 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,021 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:22,021 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:22,021 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:22,022 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:22,022 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:22,026 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:22,028 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local216527774_0019_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,029 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:22,030 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local216527774_0019_m_000000_0' done.
2015-02-10 20:24:22,030 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local216527774_0019_m_000000_0
2015-02-10 20:24:22,030 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:22,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:22,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local216527774_0019_r_000000_0
2015-02-10 20:24:22,032 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,032 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1347bc1e
2015-02-10 20:24:22,033 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:22,034 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local216527774_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:22,035 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local216527774_0019_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:22,036 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local216527774_0019_m_000000_0
2015-02-10 20:24:22,036 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:22,037 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:22,038 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,039 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:22,041 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,042 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,043 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:22,044 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:22,044 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:22,045 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,045 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,046 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,047 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:22,047 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,054 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,054 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true"]
2015-02-10 20:24:22,067 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local216527774_0019_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,069 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,069 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local216527774_0019_r_000000_0 is allowed to commit now
2015-02-10 20:24:22,070 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local216527774_0019_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true/_temporary/0/task_local216527774_0019_r_000000
2015-02-10 20:24:22,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:22,071 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local216527774_0019_r_000000_0' done.
2015-02-10 20:24:22,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local216527774_0019_r_000000_0
2015-02-10 20:24:22,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:22,093 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true/_temporary
2015-02-10 20:24:22,119 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,136 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:22,137 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:22,144 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:22,146 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:22,146 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:22,151 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:22,151 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,152 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true"]
2015-02-10 20:24:22,152 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:22,152 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:22,152 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:22,153 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/true_false_false_true
2015-02-10 20:24:22,153 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,154 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,178 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,195 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:22,210 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1138906061_0020
2015-02-10 20:24:22,224 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1138906061/.staging/job_local1138906061_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,224 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1138906061/.staging/job_local1138906061_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,265 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1138906061_0020/job_local1138906061_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,266 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1138906061_0020/job_local1138906061_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,267 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:22,268 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:22,268 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:22,268 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1138906061_0020
2015-02-10 20:24:22,269 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:22,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:22,271 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1138906061_0020_m_000000_0
2015-02-10 20:24:22,272 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,274 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:22,274 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2653a1b3
2015-02-10 20:24:22,275 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:22,275 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:22,298 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:22,298 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:22,298 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:22,299 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:22,299 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:22,300 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:22,300 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,314 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:22,314 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:22,314 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:22,315 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:22,315 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:22,319 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:22,321 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1138906061_0020_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,322 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:22,323 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1138906061_0020_m_000000_0' done.
2015-02-10 20:24:22,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1138906061_0020_m_000000_0
2015-02-10 20:24:22,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:22,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:22,324 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1138906061_0020_r_000000_0
2015-02-10 20:24:22,325 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,325 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33d02ffb
2015-02-10 20:24:22,326 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:22,328 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1138906061_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:22,330 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local1138906061_0020_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:22,331 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1138906061_0020_m_000000_0
2015-02-10 20:24:22,331 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:22,331 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:22,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,333 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:22,336 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,336 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,338 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:22,338 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:22,338 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:22,338 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,339 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,339 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,340 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:22,341 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,349 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,350 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true"]
2015-02-10 20:24:22,363 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1138906061_0020_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,364 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1138906061_0020_r_000000_0 is allowed to commit now
2015-02-10 20:24:22,365 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1138906061_0020_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true/_temporary/0/task_local1138906061_0020_r_000000
2015-02-10 20:24:22,366 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:22,366 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1138906061_0020_r_000000_0' done.
2015-02-10 20:24:22,367 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1138906061_0020_r_000000_0
2015-02-10 20:24:22,367 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:22,386 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true/_temporary
2015-02-10 20:24:22,412 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,429 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:22,429 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:22,435 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:22,437 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:22,438 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:22,443 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:22,443 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,443 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true"]
2015-02-10 20:24:22,443 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:22,444 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:22,444 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:22,444 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_true_false_true
2015-02-10 20:24:22,445 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,446 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,487 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,505 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:22,519 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1981224027_0021
2015-02-10 20:24:22,532 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1981224027/.staging/job_local1981224027_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,533 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1981224027/.staging/job_local1981224027_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,572 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981224027_0021/job_local1981224027_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,573 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981224027_0021/job_local1981224027_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,574 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:22,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:22,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:22,575 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1981224027_0021
2015-02-10 20:24:22,576 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:22,578 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:22,578 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1981224027_0021_m_000000_0
2015-02-10 20:24:22,580 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,581 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:22,581 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@42e9485
2015-02-10 20:24:22,582 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:22,582 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:22,605 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:22,605 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:22,606 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:22,606 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:22,606 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:22,608 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:22,608 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,616 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,616 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,622 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:22,622 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:22,622 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:22,622 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:22,622 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:22,626 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:22,628 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1981224027_0021_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:22,630 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981224027_0021_m_000000_0' done.
2015-02-10 20:24:22,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1981224027_0021_m_000000_0
2015-02-10 20:24:22,630 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:22,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:22,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1981224027_0021_r_000000_0
2015-02-10 20:24:22,632 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,633 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2be8ac6f
2015-02-10 20:24:22,633 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:22,634 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1981224027_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:22,636 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local1981224027_0021_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:22,636 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1981224027_0021_m_000000_0
2015-02-10 20:24:22,636 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:22,637 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:22,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,638 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:22,641 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,641 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,642 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:22,643 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:22,643 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:22,643 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,643 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,644 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,645 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:22,645 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,651 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,652 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true"]
2015-02-10 20:24:22,665 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1981224027_0021_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,667 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,667 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1981224027_0021_r_000000_0 is allowed to commit now
2015-02-10 20:24:22,667 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1981224027_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true/_temporary/0/task_local1981224027_0021_r_000000
2015-02-10 20:24:22,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:22,669 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981224027_0021_r_000000_0' done.
2015-02-10 20:24:22,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1981224027_0021_r_000000_0
2015-02-10 20:24:22,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:22,692 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true/_temporary
2015-02-10 20:24:22,718 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,737 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:22,737 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:22,744 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:22,746 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:22,747 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:22,753 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:22,754 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,755 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true"]
2015-02-10 20:24:22,755 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:22,756 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:22,756 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:22,757 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_true_false_true
2015-02-10 20:24:22,758 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,759 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:22,782 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:22,801 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:22,817 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local797759454_0022
2015-02-10 20:24:22,830 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity797759454/.staging/job_local797759454_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,831 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity797759454/.staging/job_local797759454_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,870 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local797759454_0022/job_local797759454_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:22,870 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local797759454_0022/job_local797759454_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:22,871 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:22,872 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local797759454_0022
2015-02-10 20:24:22,872 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:22,872 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:22,873 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:22,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:22,875 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local797759454_0022_m_000000_0
2015-02-10 20:24:22,877 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,878 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:22,879 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@251241f5
2015-02-10 20:24:22,880 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:22,880 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:22,903 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:22,903 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:22,903 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:22,903 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:22,903 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:22,905 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:22,905 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,914 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:22,914 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,920 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:22,920 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:22,920 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:22,920 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:22,920 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:22,924 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:22,926 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local797759454_0022_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:22,928 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local797759454_0022_m_000000_0' done.
2015-02-10 20:24:22,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local797759454_0022_m_000000_0
2015-02-10 20:24:22,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:22,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:22,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local797759454_0022_r_000000_0
2015-02-10 20:24:22,931 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:22,931 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@44215571
2015-02-10 20:24:22,932 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:22,932 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local797759454_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:22,934 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local797759454_0022_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:22,935 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local797759454_0022_m_000000_0
2015-02-10 20:24:22,935 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:22,936 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:22,937 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,937 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:22,939 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,939 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,941 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:22,941 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:22,942 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:22,942 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:22,942 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:22,943 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,943 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:22,943 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:22,950 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:22,950 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true"]
2015-02-10 20:24:22,963 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local797759454_0022_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:22,964 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:22,964 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local797759454_0022_r_000000_0 is allowed to commit now
2015-02-10 20:24:22,965 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local797759454_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true/_temporary/0/task_local797759454_0022_r_000000
2015-02-10 20:24:22,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:22,966 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local797759454_0022_r_000000_0' done.
2015-02-10 20:24:22,966 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local797759454_0022_r_000000_0
2015-02-10 20:24:22,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:22,981 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true/_temporary
2015-02-10 20:24:23,008 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,025 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:23,025 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:23,031 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:23,033 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:23,033 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:23,038 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:23,039 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,039 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true"]
2015-02-10 20:24:23,039 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:23,039 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:23,039 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:23,040 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...ive/false_false_true_true
2015-02-10 20:24:23,040 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,043 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,068 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,086 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:23,100 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1284630166_0023
2015-02-10 20:24:23,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1284630166/.staging/job_local1284630166_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,114 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1284630166/.staging/job_local1284630166_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,153 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1284630166_0023/job_local1284630166_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,154 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1284630166_0023/job_local1284630166_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,155 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:23,155 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:23,156 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:23,156 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local1284630166_0023
2015-02-10 20:24:23,156 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:23,158 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:23,158 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1284630166_0023_m_000000_0
2015-02-10 20:24:23,160 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,161 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:23,161 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3fc3530c
2015-02-10 20:24:23,162 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:23,163 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:23,185 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:23,186 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:23,186 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:23,186 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:23,186 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:23,187 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:23,187 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,194 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,195 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,200 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:23,201 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:23,201 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:23,201 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:23,201 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:23,205 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:23,206 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1284630166_0023_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,208 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:23,208 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1284630166_0023_m_000000_0' done.
2015-02-10 20:24:23,208 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1284630166_0023_m_000000_0
2015-02-10 20:24:23,209 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:23,209 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:23,209 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1284630166_0023_r_000000_0
2015-02-10 20:24:23,211 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,211 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@32932bc1
2015-02-10 20:24:23,212 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:23,212 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1284630166_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:23,214 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local1284630166_0023_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:23,215 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local1284630166_0023_m_000000_0
2015-02-10 20:24:23,215 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:23,216 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:23,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,217 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:23,218 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,219 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,220 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:23,220 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:23,221 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:23,221 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,221 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,222 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,222 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:23,222 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,228 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,229 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true"]
2015-02-10 20:24:23,241 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1284630166_0023_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,243 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,243 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1284630166_0023_r_000000_0 is allowed to commit now
2015-02-10 20:24:23,244 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1284630166_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true/_temporary/0/task_local1284630166_0023_r_000000
2015-02-10 20:24:23,245 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:23,245 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1284630166_0023_r_000000_0' done.
2015-02-10 20:24:23,246 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1284630166_0023_r_000000_0
2015-02-10 20:24:23,246 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:23,261 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true/_temporary
2015-02-10 20:24:23,288 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,305 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:23,305 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:23,313 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:23,315 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:23,316 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:23,321 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:23,321 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,321 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true"]
2015-02-10 20:24:23,322 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:23,322 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:23,322 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:23,323 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/true_false_true_true
2015-02-10 20:24:23,323 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,325 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,348 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,395 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:23,410 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local615447346_0024
2015-02-10 20:24:23,423 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity615447346/.staging/job_local615447346_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,424 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity615447346/.staging/job_local615447346_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,463 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local615447346_0024/job_local615447346_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,463 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local615447346_0024/job_local615447346_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,464 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:23,465 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local615447346_0024
2015-02-10 20:24:23,465 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:23,465 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:23,465 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:23,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:23,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local615447346_0024_m_000000_0
2015-02-10 20:24:23,470 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,471 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:23,471 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@237dc815
2015-02-10 20:24:23,472 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:23,472 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:23,496 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:23,496 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:23,496 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:23,496 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:23,497 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:23,498 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:23,498 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,508 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,508 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:23,514 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:23,514 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:23,514 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:23,514 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:23,518 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:23,520 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local615447346_0024_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,522 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:23,522 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local615447346_0024_m_000000_0' done.
2015-02-10 20:24:23,522 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local615447346_0024_m_000000_0
2015-02-10 20:24:23,522 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:23,523 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:23,523 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local615447346_0024_r_000000_0
2015-02-10 20:24:23,525 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,525 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6dbaa46d
2015-02-10 20:24:23,526 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:23,527 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local615447346_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:23,530 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local615447346_0024_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:23,530 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local615447346_0024_m_000000_0
2015-02-10 20:24:23,531 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:23,532 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:23,533 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,533 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:23,535 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,536 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,538 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:23,538 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:23,538 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:23,539 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,539 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,540 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,541 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:23,541 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,548 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,548 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true"]
2015-02-10 20:24:23,561 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local615447346_0024_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,563 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local615447346_0024_r_000000_0 is allowed to commit now
2015-02-10 20:24:23,563 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local615447346_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true/_temporary/0/task_local615447346_0024_r_000000
2015-02-10 20:24:23,564 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:23,565 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local615447346_0024_r_000000_0' done.
2015-02-10 20:24:23,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local615447346_0024_r_000000_0
2015-02-10 20:24:23,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:23,583 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true/_temporary
2015-02-10 20:24:23,609 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,626 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:23,626 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:23,632 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:23,633 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:23,634 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:23,638 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:23,638 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,639 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true"]
2015-02-10 20:24:23,639 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:23,639 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:23,639 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:23,640 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...nsive/true_true_true_true
2015-02-10 20:24:23,642 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,643 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,668 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,686 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:23,701 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local221363273_0025
2015-02-10 20:24:23,714 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity221363273/.staging/job_local221363273_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,714 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity221363273/.staging/job_local221363273_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,753 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local221363273_0025/job_local221363273_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:23,753 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local221363273_0025/job_local221363273_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:23,754 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:23,754 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local221363273_0025
2015-02-10 20:24:23,754 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:23,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:23,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:23,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:23,759 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local221363273_0025_m_000000_0
2015-02-10 20:24:23,760 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,761 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:23,761 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@440149a7
2015-02-10 20:24:23,763 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:23,763 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:23,786 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:23,786 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:23,786 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:23,786 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:23,786 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:23,788 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:23,788 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,796 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,796 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:23,802 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:23,802 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:23,803 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:23,803 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:23,807 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:23,809 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local221363273_0025_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:23,811 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local221363273_0025_m_000000_0' done.
2015-02-10 20:24:23,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local221363273_0025_m_000000_0
2015-02-10 20:24:23,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:23,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:23,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local221363273_0025_r_000000_0
2015-02-10 20:24:23,814 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:23,814 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@615a8006
2015-02-10 20:24:23,815 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:23,815 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local221363273_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:23,817 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local221363273_0025_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:23,818 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local221363273_0025_m_000000_0
2015-02-10 20:24:23,818 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:23,819 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:23,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,820 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:23,822 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,823 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,824 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:23,825 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:23,825 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:23,825 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:23,825 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:23,826 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,827 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:23,827 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:23,833 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:23,833 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true"]
2015-02-10 20:24:23,846 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local221363273_0025_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:23,848 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:23,848 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local221363273_0025_r_000000_0 is allowed to commit now
2015-02-10 20:24:23,849 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local221363273_0025_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true/_temporary/0/task_local221363273_0025_r_000000
2015-02-10 20:24:23,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:23,850 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local221363273_0025_r_000000_0' done.
2015-02-10 20:24:23,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local221363273_0025_r_000000_0
2015-02-10 20:24:23,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:23,873 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true/_temporary
2015-02-10 20:24:23,899 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,916 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:24:23,917 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:24:23,923 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:24:23,925 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:24:23,925 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:24:23,929 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting
2015-02-10 20:24:23,930 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] source: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:23,930 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] sink: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true"]
2015-02-10 20:24:23,930 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] parallel execution is enabled: false
2015-02-10 20:24:23,930 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] starting jobs: 1
2015-02-10 20:24:23,930 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [comprehensivesort] allocating threads: 1
2015-02-10 20:24:23,931 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] starting step: (1/1) ...sive/false_true_true_true
2015-02-10 20:24:23,931 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,932 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:24:23,956 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:23,974 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:24:23,988 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local155196161_0026
2015-02-10 20:24:24,003 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity155196161/.staging/job_local155196161_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:24,003 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity155196161/.staging/job_local155196161_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:24,043 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local155196161_0026/job_local155196161_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 20:24:24,043 WARN conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local155196161_0026/job_local155196161_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 20:24:24,045 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:24:24,047 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] submitted hadoop job: job_local155196161_0026
2015-02-10 20:24:24,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:24:24,047 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [comprehensivesort] tracking url: http://localhost:8080/
2015-02-10 20:24:24,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:24:24,050 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:24:24,050 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local155196161_0026_m_000000_0
2015-02-10 20:24:24,052 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:24,053 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:24:24,053 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1495c8c2
2015-02-10 20:24:24,054 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:24:24,055 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:24:24,078 INFO mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:24:24,078 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:24:24,078 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:24:24,079 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:24:24,079 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:24:24,081 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:24:24,081 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:24:24,089 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:24:24,089 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:24,095 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 20:24:24,095 INFO mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:24:24,096 INFO mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:24:24,096 INFO mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 888; bufvoid = 104857600
2015-02-10 20:24:24,096 INFO mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214252(104857008); length = 145/6553600
2015-02-10 20:24:24,101 INFO mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:24:24,102 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local155196161_0026_m_000000_0 is done. And is in the process of committing
2015-02-10 20:24:24,104 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:24:24,104 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local155196161_0026_m_000000_0' done.
2015-02-10 20:24:24,105 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local155196161_0026_m_000000_0
2015-02-10 20:24:24,105 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:24:24,105 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:24:24,106 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local155196161_0026_r_000000_0
2015-02-10 20:24:24,107 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:24:24,107 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6acbc2da
2015-02-10 20:24:24,108 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:24:24,109 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local155196161_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:24:24,110 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local155196161_0026_m_000000_0 decomp: 964 len: 968 to MEMORY
2015-02-10 20:24:24,111 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 964 bytes from map-output for attempt_local155196161_0026_m_000000_0
2015-02-10 20:24:24,111 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 964, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->964
2015-02-10 20:24:24,112 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:24:24,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:24,114 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:24:24,120 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:24,120 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:24,122 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 964 bytes to disk to satisfy reduce memory limit
2015-02-10 20:24:24,122 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 968 bytes from disk
2015-02-10 20:24:24,123 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:24:24,123 INFO mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:24:24,123 INFO mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 952 bytes
2015-02-10 20:24:24,124 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:24,124 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:24:24,125 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:24:24,132 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(comprehensivesort)[by:[{1}:'num']]
2015-02-10 20:24:24,132 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true"]
2015-02-10 20:24:24,147 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local155196161_0026_r_000000_0 is done. And is in the process of committing
2015-02-10 20:24:24,148 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:24:24,148 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local155196161_0026_r_000000_0 is allowed to commit now
2015-02-10 20:24:24,149 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local155196161_0026_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true/_temporary/0/task_local155196161_0026_r_000000
2015-02-10 20:24:24,150 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:24:24,150 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local155196161_0026_r_000000_0' done.
2015-02-10 20:24:24,151 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local155196161_0026_r_000000_0
2015-02-10 20:24:24,151 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:24:24,165 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true/_temporary
2015-02-10 20:24:24,193 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:24:24,196 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_true
2015-02-10 20:24:24,197 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_true
2015-02-10 20:24:24,197 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_true
2015-02-10 20:24:24,197 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_true
2015-02-10 20:24:24,197 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_false
2015-02-10 20:24:24,197 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_false
2015-02-10 20:24:24,198 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_true_true
2015-02-10 20:24:24,198 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_true
2015-02-10 20:24:24,198 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_true
2015-02-10 20:24:24,198 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_true_false
2015-02-10 20:24:24,199 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_false
2015-02-10 20:24:24,199 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_false_false
2015-02-10 20:24:24,199 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_false_false
2015-02-10 20:24:24,199 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_true_false
2015-02-10 20:24:24,199 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_false
2015-02-10 20:24:24,200 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_false
2015-02-10 20:24:24,200 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_false_true_true
2015-02-10 20:24:24,200 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_true_false_true
2015-02-10 20:24:24,200 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_null_false_true
2015-02-10 20:24:24,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/true_false_false_false
2015-02-10 20:24:24,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_true
2015-02-10 20:24:24,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_true_true_false
2015-02-10 20:24:24,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_true_false
2015-02-10 20:24:24,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/sortedvaluesplatform/comprehensive/false_null_false_true