2015-02-10 22:48:41,284 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-10 22:48:41,352 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-10 22:48:41,352 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2015-02-10 22:48:41,400 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-10 22:48:42,045 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-10 22:48:42,650 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:42,653 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:42,665 INFO property.AppProps (AppProps.java:getAppID(169)) - using app.id: 3C067E13787C4D43B366892043BA8958
2015-02-10 22:48:42,764 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:42,806 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:42,809 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:42,960 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-10 22:48:42,962 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:42,963 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:42,964 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:42,964 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 22:48:42,966 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 22:48:42,966 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 22:48:42,966 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 22:48:42,968 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2015-02-10 22:48:42,992 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-10 22:48:42,993 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-10 22:48:43,021 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:43,346 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:43,358 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:43,389 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:43,550 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local782323639_0001
2015-02-10 22:48:43,592 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity782323639/.staging/job_local782323639_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:43,605 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity782323639/.staging/job_local782323639_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:43,795 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local782323639_0001/job_local782323639_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:43,803 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local782323639_0001/job_local782323639_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:43,810 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:43,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:43,815 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local782323639_0001
2015-02-10 22:48:43,816 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:43,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:43,888 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:43,891 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:43,936 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:43,946 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:43,946 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@565539d8
2015-02-10 22:48:43,955 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:43,962 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:44,124 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:44,124 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:44,124 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:44,124 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:44,125 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:44,154 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:44,155 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,218 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-10 22:48:44,226 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:44,227 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,230 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:44,245 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:44,246 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,246 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,247 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:44,248 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,248 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,249 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:44,249 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,249 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,250 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:44,250 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 202, max: 672
2015-02-10 22:48:44,251 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,255 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:44,255 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:44,255 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:44,255 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:44,256 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:44,270 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:44,274 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:44,286 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:44,288 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_m_000000_0' done.
2015-02-10 22:48:44,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:44,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,291 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:44,293 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:44,293 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@c58f769
2015-02-10 22:48:44,295 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:44,295 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:44,449 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:44,450 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:44,450 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:44,450 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:44,450 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:44,454 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:44,454 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,481 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:44,481 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,482 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:44,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:44,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:44,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,580 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,582 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:44,582 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,582 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,583 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:44,583 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 83, total: 302, max: 672
2015-02-10 22:48:44,583 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:44,585 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:44,585 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:44,586 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:44,586 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:44,586 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:44,592 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:44,595 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:44,598 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:44,598 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_m_000001_0' done.
2015-02-10 22:48:44,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:44,602 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:44,602 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local782323639_0001_r_000000_0
2015-02-10 22:48:44,610 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:44,613 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1b435686
2015-02-10 22:48:44,627 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:44,631 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local782323639_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:44,661 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local782323639_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:44,666 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local782323639_0001_m_000001_0
2015-02-10 22:48:44,708 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 22:48:44,712 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local782323639_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:44,713 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local782323639_0001_m_000000_0
2015-02-10 22:48:44,713 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 22:48:44,714 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:44,715 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,716 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:44,728 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:44,729 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 22:48:44,732 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:44,733 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 22:48:44,733 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:44,734 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:44,734 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 22:48:44,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,738 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:44,738 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:44,765 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 22:48:44,766 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 22:48:44,795 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local782323639_0001_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:44,797 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:44,798 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local782323639_0001_r_000000_0 is allowed to commit now
2015-02-10 22:48:44,799 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local782323639_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local782323639_0001_r_000000
2015-02-10 22:48:44,800 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:44,801 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local782323639_0001_r_000000_0' done.
2015-02-10 22:48:44,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local782323639_0001_r_000000_0
2015-02-10 22:48:44,801 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:44,867 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2015-02-10 22:48:44,916 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:44,961 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:44,965 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2015-02-10 22:48:45,003 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:45,004 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:45,022 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:45,028 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:45,029 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:45,047 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:45,047 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,048 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 22:48:45,048 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 22:48:45,048 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 22:48:45,050 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 22:48:45,051 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2015-02-10 22:48:45,052 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,054 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,087 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,114 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:45,150 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local975825011_0002
2015-02-10 22:48:45,177 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity975825011/.staging/job_local975825011_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:45,181 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity975825011/.staging/job_local975825011_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:45,282 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local975825011_0002/job_local975825011_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:45,286 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local975825011_0002/job_local975825011_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:45,287 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:45,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:45,288 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:45,288 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local975825011_0002
2015-02-10 22:48:45,290 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:45,293 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:45,293 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,296 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:45,298 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:45,298 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@388eed76
2015-02-10 22:48:45,302 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:45,302 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:45,452 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:45,452 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:45,452 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:45,452 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:45,452 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:45,455 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:45,455 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:45,469 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,469 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:45,470 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:45,472 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:45,472 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 403, max: 672
2015-02-10 22:48:45,473 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:45,473 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:45,474 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 88, total: 403, max: 672
2015-02-10 22:48:45,474 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:45,476 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:45,477 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:45,477 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:45,477 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:48:45,477 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:45,482 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:45,485 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local975825011_0002_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:45,488 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:45,488 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local975825011_0002_m_000000_0' done.
2015-02-10 22:48:45,488 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,489 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:45,489 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:45,490 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local975825011_0002_r_000000_0
2015-02-10 22:48:45,492 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:45,493 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@32552379
2015-02-10 22:48:45,495 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:45,496 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local975825011_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:45,499 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local975825011_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:48:45,500 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local975825011_0002_m_000000_0
2015-02-10 22:48:45,500 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:48:45,501 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:45,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,503 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:45,507 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:45,508 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:45,510 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:45,511 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:48:45,511 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:45,511 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:45,512 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:45,514 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,515 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:45,515 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:45,529 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:45,529 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 22:48:45,546 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local975825011_0002_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:45,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:45,548 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local975825011_0002_r_000000_0 is allowed to commit now
2015-02-10 22:48:45,549 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local975825011_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local975825011_0002_r_000000
2015-02-10 22:48:45,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:45,551 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local975825011_0002_r_000000_0' done.
2015-02-10 22:48:45,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local975825011_0002_r_000000_0
2015-02-10 22:48:45,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:45,581 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2015-02-10 22:48:45,625 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,669 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,671 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2015-02-10 22:48:45,725 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:45,726 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:45,749 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:45,757 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:45,758 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:45,773 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting
2015-02-10 22:48:45,774 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:45,774 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:45,774 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 22:48:45,775 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] parallel execution is enabled: false
2015-02-10 22:48:45,775 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting jobs: 1
2015-02-10 22:48:45,775 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] allocating threads: 1
2015-02-10 22:48:45,776 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2015-02-10 22:48:45,777 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,778 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:45,811 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,814 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:45,839 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:45,879 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local133005498_0003
2015-02-10 22:48:45,905 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity133005498/.staging/job_local133005498_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:45,908 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity133005498/.staging/job_local133005498_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:46,000 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local133005498_0003/job_local133005498_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:46,003 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local133005498_0003/job_local133005498_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:46,004 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:46,004 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local133005498_0003
2015-02-10 22:48:46,005 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:46,005 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:46,006 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:46,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:46,011 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,015 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,018 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:46,018 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2eec0962
2015-02-10 22:48:46,021 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:46,021 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:46,170 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:46,171 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:46,171 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:46,171 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:46,171 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:46,174 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,174 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,191 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:46,191 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,192 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:46,194 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:46,194 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,194 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,195 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:46,195 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,195 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,196 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:46,196 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,197 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,197 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:46,197 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 122, total: 529, max: 672
2015-02-10 22:48:46,197 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 78% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:46,199 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:46,200 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:46,200 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:46,200 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:46,200 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:46,205 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:46,208 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,211 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:46,211 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_m_000000_0' done.
2015-02-10 22:48:46,211 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,211 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,214 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,216 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:46,216 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12072edc
2015-02-10 22:48:46,217 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:46,218 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:46,364 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:46,364 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:46,364 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:46,364 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:46,364 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:46,366 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,366 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,382 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:46,382 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,383 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:46,387 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:46,387 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:46,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:46,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:46,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 53, total: 561, max: 672
2015-02-10 22:48:46,392 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:46,392 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:46,393 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:46,393 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:46,393 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:46,398 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:46,400 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:46,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:46,403 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_m_000001_0' done.
2015-02-10 22:48:46,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:46,405 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:46,405 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local133005498_0003_r_000000_0
2015-02-10 22:48:46,408 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,408 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@54f690e4
2015-02-10 22:48:46,409 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:46,410 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local133005498_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:46,414 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local133005498_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:46,414 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local133005498_0003_m_000000_0
2015-02-10 22:48:46,415 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:46,418 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local133005498_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:46,418 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local133005498_0003_m_000001_0
2015-02-10 22:48:46,419 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2015-02-10 22:48:46,419 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:46,420 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,421 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:46,423 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:46,424 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2015-02-10 22:48:46,426 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:46,427 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 402 bytes from disk
2015-02-10 22:48:46,427 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:46,427 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:46,428 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2015-02-10 22:48:46,428 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,429 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:46,429 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,444 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 22:48:46,444 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 22:48:46,460 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local133005498_0003_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,462 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:46,462 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local133005498_0003_r_000000_0 is allowed to commit now
2015-02-10 22:48:46,465 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local133005498_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local133005498_0003_r_000000
2015-02-10 22:48:46,467 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:46,467 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local133005498_0003_r_000000_0' done.
2015-02-10 22:48:46,467 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local133005498_0003_r_000000_0
2015-02-10 22:48:46,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:46,510 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2015-02-10 22:48:46,549 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,588 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,590 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2015-02-10 22:48:46,624 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:46,625 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:46,639 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:46,643 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:46,644 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:46,659 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:46,659 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:46,659 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 22:48:46,660 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:48:46,660 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:48:46,660 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:48:46,661 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2015-02-10 22:48:46,662 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:46,663 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:46,699 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:46,724 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:46,749 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1647339523_0004
2015-02-10 22:48:46,774 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1647339523/.staging/job_local1647339523_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:46,775 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1647339523/.staging/job_local1647339523_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:46,850 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1647339523_0004/job_local1647339523_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:46,852 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1647339523_0004/job_local1647339523_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:46,853 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:46,853 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1647339523_0004
2015-02-10 22:48:46,854 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:46,854 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:46,855 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:46,860 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:46,860 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1647339523_0004_m_000000_0
2015-02-10 22:48:46,863 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:46,865 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:46,866 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@405a02ef
2015-02-10 22:48:46,867 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:46,878 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:46,878 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:46,887 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:46,887 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 22:48:46,891 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:46,891 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1647339523_0004_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:46,893 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:46,894 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1647339523_0004_m_000000_0 is allowed to commit now
2015-02-10 22:48:46,895 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1647339523_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local1647339523_0004_m_000000
2015-02-10 22:48:46,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:46,896 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1647339523_0004_m_000000_0' done.
2015-02-10 22:48:46,897 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1647339523_0004_m_000000_0
2015-02-10 22:48:46,897 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:47,243 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2015-02-10 22:48:47,280 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,283 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2015-02-10 22:48:47,318 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:47,319 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:47,344 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:47,348 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:47,369 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:47,381 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:47,382 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:47,382 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 22:48:47,382 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 22:48:47,382 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 2
2015-02-10 22:48:47,383 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 22:48:47,383 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2015-02-10 22:48:47,385 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,386 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,419 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,442 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:47,476 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1304463010_0005
2015-02-10 22:48:47,497 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1304463010/.staging/job_local1304463010_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:47,499 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1304463010/.staging/job_local1304463010_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:47,559 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1304463010_0005/job_local1304463010_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:47,561 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1304463010_0005/job_local1304463010_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:47,561 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:47,562 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1304463010_0005
2015-02-10 22:48:47,562 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:47,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:47,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:47,567 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:47,567 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,569 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,571 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:47,571 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@54182d86
2015-02-10 22:48:47,573 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:47,573 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:47,650 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:47,650 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:47,650 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:47,650 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:47,650 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:47,652 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:47,652 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:47,666 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:47,667 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:47,667 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:47,669 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:47,669 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,669 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,670 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:47,670 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:47,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:47,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 76, total: 219, max: 672
2015-02-10 22:48:47,673 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,674 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:47,674 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:47,674 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:47,675 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:47,675 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:47,680 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:47,682 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1304463010_0005_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:47,686 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:47,687 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1304463010_0005_m_000000_0' done.
2015-02-10 22:48:47,687 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,687 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:47,688 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:47,688 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1304463010_0005_r_000000_0
2015-02-10 22:48:47,690 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,690 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@25082661
2015-02-10 22:48:47,691 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:47,692 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1304463010_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:47,694 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local1304463010_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:47,695 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1304463010_0005_m_000000_0
2015-02-10 22:48:47,695 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:47,696 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:47,697 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,697 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:47,702 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:47,702 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:47,704 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:47,704 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:47,705 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:47,705 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:47,705 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:47,706 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,706 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:47,706 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:47,719 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:47,720 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][9561124881/count/]
2015-02-10 22:48:47,746 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:47,749 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:47,750 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 72, total: 219, max: 672
2015-02-10 22:48:47,750 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:47,751 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1304463010_0005_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:47,753 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:47,753 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1304463010_0005_r_000000_0 is allowed to commit now
2015-02-10 22:48:47,754 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1304463010_0005_r_000000_0' to file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/_temporary/0/task_local1304463010_0005_r_000000
2015-02-10 22:48:47,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:47,756 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1304463010_0005_r_000000_0' done.
2015-02-10 22:48:47,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1304463010_0005_r_000000_0
2015-02-10 22:48:47,757 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:47,786 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2015-02-10 22:48:47,786 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,788 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:47,832 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:47,865 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:47,879 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:47,879 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:47,887 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local167084815_0006
2015-02-10 22:48:47,912 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity167084815/.staging/job_local167084815_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:47,913 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity167084815/.staging/job_local167084815_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:47,969 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local167084815_0006/job_local167084815_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:47,970 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local167084815_0006/job_local167084815_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:47,971 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:47,972 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local167084815_0006
2015-02-10 22:48:47,972 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:47,972 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:47,973 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:47,976 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:47,977 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:47,979 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:47,981 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/part-00000
2015-02-10 22:48:47,981 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@157ee3e5
2015-02-10 22:48:47,994 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:47,995 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,154 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,155 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,155 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,155 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,155 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,157 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,157 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,168 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][9561124881/count/]
2015-02-10 22:48:48,169 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2015-02-10 22:48:48,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:48,171 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,172 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,172 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2015-02-10 22:48:48,172 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2015-02-10 22:48:48,175 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,177 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local167084815_0006_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,180 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/9561124881_count_9B0E1FAC21EB4717BBC6F58183F0801D/part-00000:0+112
2015-02-10 22:48:48,180 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local167084815_0006_m_000000_0' done.
2015-02-10 22:48:48,180 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:48,181 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:48,181 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:48,181 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local167084815_0006_r_000000_0
2015-02-10 22:48:48,183 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,184 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@296068c9
2015-02-10 22:48:48,185 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:48,187 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local167084815_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:48,189 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local167084815_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2015-02-10 22:48:48,189 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local167084815_0006_m_000000_0
2015-02-10 22:48:48,190 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2015-02-10 22:48:48,190 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:48,191 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,193 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:48,195 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,195 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 22:48:48,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:48,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 33 bytes from disk
2015-02-10 22:48:48,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:48,198 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,198 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 22:48:48,199 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,199 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:48,199 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,210 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2015-02-10 22:48:48,210 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 22:48:48,222 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local167084815_0006_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,227 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:48,227 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local167084815_0006_r_000000_0 is allowed to commit now
2015-02-10 22:48:48,229 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local167084815_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local167084815_0006_r_000000
2015-02-10 22:48:48,230 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:48,231 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local167084815_0006_r_000000_0' done.
2015-02-10 22:48:48,231 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local167084815_0006_r_000000_0
2015-02-10 22:48:48,232 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:48,256 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2015-02-10 22:48:48,290 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,326 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,329 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2015-02-10 22:48:48,362 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:48,362 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:48,382 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:48,388 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:48,388 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:48,400 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting
2015-02-10 22:48:48,400 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:48,400 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:48,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 22:48:48,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] parallel execution is enabled: false
2015-02-10 22:48:48,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting jobs: 1
2015-02-10 22:48:48,401 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] allocating threads: 1
2015-02-10 22:48:48,402 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2015-02-10 22:48:48,403 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:48,404 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:48,433 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,435 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:48,456 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:48,475 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local882716914_0007
2015-02-10 22:48:48,503 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity882716914/.staging/job_local882716914_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:48,505 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity882716914/.staging/job_local882716914_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:48,560 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882716914_0007/job_local882716914_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:48,562 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882716914_0007/job_local882716914_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:48,563 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:48,563 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local882716914_0007
2015-02-10 22:48:48,564 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:48,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:48,564 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:48,569 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:48,569 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,571 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,573 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:48,573 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@245f4ae
2015-02-10 22:48:48,575 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:48,575 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,725 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,725 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,725 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,726 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,726 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,727 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,727 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:48,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:48,742 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:48,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:48,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:48,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:48,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,747 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:48,747 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 432, max: 672
2015-02-10 22:48:48,747 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,749 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:48,749 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,749 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,749 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:48,749 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:48,754 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,756 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:48,759 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:48,759 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_m_000000_0' done.
2015-02-10 22:48:48,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,762 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,764 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:48,764 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48ec77cb
2015-02-10 22:48:48,765 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:48,765 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:48,911 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:48,911 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:48,911 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:48,911 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:48,912 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:48,913 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:48,913 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:48,927 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:48,927 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:48,927 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:48,932 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:48,932 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,932 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,933 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:48,933 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,933 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:48,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:48,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 112, total: 532, max: 672
2015-02-10 22:48:48,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:48,937 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:48,937 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:48,937 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:48,937 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:48,937 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:48,941 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:48,944 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:48,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:48,946 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_m_000001_0' done.
2015-02-10 22:48:48,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,947 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:48,947 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:48,947 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local882716914_0007_r_000000_0
2015-02-10 22:48:48,950 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:48,950 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@c88a80e
2015-02-10 22:48:48,951 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:48,952 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local882716914_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:48,956 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local882716914_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:48,957 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882716914_0007_m_000000_0
2015-02-10 22:48:48,957 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:48,960 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local882716914_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:48,960 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882716914_0007_m_000001_0
2015-02-10 22:48:48,960 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 22:48:48,961 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:48,962 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:48,963 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:48,991 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:48,991 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 22:48:48,993 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:48,994 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 22:48:48,994 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:48,994 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:48,994 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 22:48:48,995 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:48,995 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:48,996 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,009 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:49,009 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 22:48:49,022 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local882716914_0007_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,024 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:49,024 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local882716914_0007_r_000000_0 is allowed to commit now
2015-02-10 22:48:49,025 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local882716914_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local882716914_0007_r_000000
2015-02-10 22:48:49,026 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:49,026 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882716914_0007_r_000000_0' done.
2015-02-10 22:48:49,027 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local882716914_0007_r_000000_0
2015-02-10 22:48:49,027 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:49,052 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2015-02-10 22:48:49,087 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,121 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,123 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2015-02-10 22:48:49,151 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:49,151 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:49,163 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:49,166 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:49,166 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:49,176 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 22:48:49,176 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,177 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 22:48:49,177 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] parallel execution is enabled: false
2015-02-10 22:48:49,177 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting jobs: 1
2015-02-10 22:48:49,178 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] allocating threads: 1
2015-02-10 22:48:49,179 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...splatform/minbynullsafety
2015-02-10 22:48:49,179 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,181 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,214 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,235 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:49,254 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local804643557_0008
2015-02-10 22:48:49,274 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity804643557/.staging/job_local804643557_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:49,276 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity804643557/.staging/job_local804643557_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:49,348 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local804643557_0008/job_local804643557_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:49,350 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local804643557_0008/job_local804643557_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:49,352 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:49,352 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local804643557_0008
2015-02-10 22:48:49,352 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 22:48:49,352 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:49,353 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:49,357 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:49,357 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,360 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:49,362 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:49,362 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2cec849e
2015-02-10 22:48:49,364 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:49,365 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:49,696 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:49,697 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:49,698 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:49,698 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:49,698 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:49,700 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:49,700 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,712 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,712 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:49,713 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:49,715 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:49,716 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 152, total: 257, max: 672
2015-02-10 22:48:49,716 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,717 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:49,718 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,718 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,719 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:49,719 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,720 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,721 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:49,721 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 151, total: 257, max: 672
2015-02-10 22:48:49,721 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:49,724 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:49,725 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:49,725 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:49,726 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:49,726 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:49,730 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:49,733 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local804643557_0008_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:49,736 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local804643557_0008_m_000000_0' done.
2015-02-10 22:48:49,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,737 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:49,737 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:49,738 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local804643557_0008_r_000000_0
2015-02-10 22:48:49,740 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:49,740 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7417d56b
2015-02-10 22:48:49,741 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:49,742 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local804643557_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:49,744 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local804643557_0008_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:49,745 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local804643557_0008_m_000000_0
2015-02-10 22:48:49,745 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:49,746 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:49,748 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,748 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:49,751 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:49,751 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:49,753 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:49,754 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:49,754 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:49,754 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:49,755 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:49,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,757 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:49,757 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:49,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:49,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 22:48:49,780 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local804643557_0008_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:49,782 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:49,783 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local804643557_0008_r_000000_0 is allowed to commit now
2015-02-10 22:48:49,784 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local804643557_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary/0/task_local804643557_0008_r_000000
2015-02-10 22:48:49,785 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:49,786 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local804643557_0008_r_000000_0' done.
2015-02-10 22:48:49,786 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local804643557_0008_r_000000_0
2015-02-10 22:48:49,787 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:49,809 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary
2015-02-10 22:48:49,842 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,875 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,878 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbynullsafety
2015-02-10 22:48:49,903 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:49,903 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:49,913 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:49,915 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:49,916 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:49,926 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:49,927 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:49,927 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 22:48:49,927 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 22:48:49,927 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 22:48:49,927 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 22:48:49,928 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2015-02-10 22:48:49,928 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,930 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:49,957 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:49,977 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:49,995 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local808692111_0009
2015-02-10 22:48:50,014 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity808692111/.staging/job_local808692111_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:50,015 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity808692111/.staging/job_local808692111_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:50,068 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local808692111_0009/job_local808692111_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:50,069 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local808692111_0009/job_local808692111_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:50,070 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:50,070 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local808692111_0009
2015-02-10 22:48:50,070 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:50,071 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:50,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:50,074 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:50,075 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,076 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,078 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:50,078 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ef4b65d
2015-02-10 22:48:50,080 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:50,080 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:50,230 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:50,231 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:50,231 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:50,231 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:50,231 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:50,232 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:50,233 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,243 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:50,243 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:50,244 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:50,246 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:50,246 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 105, total: 357, max: 672
2015-02-10 22:48:50,246 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 53% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,247 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:50,247 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 105, total: 357, max: 672
2015-02-10 22:48:50,247 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 53% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:50,249 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:50,249 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:50,249 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:48:50,249 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:50,252 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:50,254 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local808692111_0009_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:50,257 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local808692111_0009_m_000000_0' done.
2015-02-10 22:48:50,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:50,258 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:50,258 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local808692111_0009_r_000000_0
2015-02-10 22:48:50,260 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,260 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@669aa3f3
2015-02-10 22:48:50,261 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:50,262 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local808692111_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:50,264 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local808692111_0009_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:48:50,264 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local808692111_0009_m_000000_0
2015-02-10 22:48:50,264 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:48:50,265 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:50,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,266 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:50,268 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,269 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:50,270 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:50,271 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:48:50,271 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:50,271 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,271 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:48:50,272 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,272 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:50,273 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,283 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 22:48:50,283 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 22:48:50,295 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local808692111_0009_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,297 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,297 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local808692111_0009_r_000000_0 is allowed to commit now
2015-02-10 22:48:50,298 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local808692111_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local808692111_0009_r_000000
2015-02-10 22:48:50,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:50,299 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local808692111_0009_r_000000_0' done.
2015-02-10 22:48:50,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local808692111_0009_r_000000_0
2015-02-10 22:48:50,299 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:50,323 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2015-02-10 22:48:50,356 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,393 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,396 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2015-02-10 22:48:50,433 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:50,434 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:50,446 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:50,449 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:50,451 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:50,467 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 22:48:50,468 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 22:48:50,468 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 22:48:50,468 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] parallel execution is enabled: false
2015-02-10 22:48:50,468 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting jobs: 1
2015-02-10 22:48:50,468 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] allocating threads: 1
2015-02-10 22:48:50,469 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2015-02-10 22:48:50,470 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:50,471 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:50,501 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,525 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:50,552 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2092494425_0010
2015-02-10 22:48:50,578 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2092494425/.staging/job_local2092494425_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:50,580 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2092494425/.staging/job_local2092494425_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:50,650 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2092494425_0010/job_local2092494425_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:50,652 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2092494425_0010/job_local2092494425_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:50,653 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:50,653 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local2092494425_0010
2015-02-10 22:48:50,654 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:50,654 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 22:48:50,654 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:50,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:50,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,659 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,661 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 22:48:50,661 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@74a17378
2015-02-10 22:48:50,663 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:50,663 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:50,811 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:50,812 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:50,813 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:50,813 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:50,813 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:50,817 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:50,818 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,829 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 22:48:50,829 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:50,830 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:50,834 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:50,834 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 140, total: 457, max: 672
2015-02-10 22:48:50,835 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,836 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:50,837 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 140, total: 457, max: 672
2015-02-10 22:48:50,837 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 68% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:50,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:50,840 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:50,840 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:50,841 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 170; bufvoid = 104857600
2015-02-10 22:48:50,841 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:50,844 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:50,847 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2092494425_0010_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,849 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 22:48:50,849 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2092494425_0010_m_000000_0' done.
2015-02-10 22:48:50,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,850 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:50,851 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:50,851 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2092494425_0010_r_000000_0
2015-02-10 22:48:50,853 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:50,854 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@72ea9cf9
2015-02-10 22:48:50,854 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:50,855 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2092494425_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:50,858 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local2092494425_0010_m_000000_0 decomp: 182 len: 186 to MEMORY
2015-02-10 22:48:50,858 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 182 bytes from map-output for attempt_local2092494425_0010_m_000000_0
2015-02-10 22:48:50,858 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 182, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->182
2015-02-10 22:48:50,859 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:50,860 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,860 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:50,863 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,864 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 22:48:50,866 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 182 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:50,866 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 186 bytes from disk
2015-02-10 22:48:50,867 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:50,867 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:50,867 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 22:48:50,868 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,869 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:50,870 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:50,879 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:50,880 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 22:48:50,891 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2092494425_0010_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:50,893 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:50,893 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2092494425_0010_r_000000_0 is allowed to commit now
2015-02-10 22:48:50,894 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2092494425_0010_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary/0/task_local2092494425_0010_r_000000
2015-02-10 22:48:50,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:50,896 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2092494425_0010_r_000000_0' done.
2015-02-10 22:48:50,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2092494425_0010_r_000000_0
2015-02-10 22:48:50,897 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:50,916 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary
2015-02-10 22:48:50,947 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:50,949 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfields
2015-02-10 22:48:50,975 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:50,976 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:50,984 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:50,986 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:50,987 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:50,996 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 22:48:50,996 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:50,996 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 22:48:50,997 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] parallel execution is enabled: false
2015-02-10 22:48:50,997 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting jobs: 1
2015-02-10 22:48:50,997 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] allocating threads: 1
2015-02-10 22:48:50,999 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2015-02-10 22:48:50,999 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,001 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,025 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,045 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,062 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1359104465_0011
2015-02-10 22:48:51,083 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1359104465/.staging/job_local1359104465_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,084 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1359104465/.staging/job_local1359104465_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,138 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1359104465_0011/job_local1359104465_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,139 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1359104465_0011/job_local1359104465_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,140 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,140 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1359104465_0011
2015-02-10 22:48:51,141 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,141 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 22:48:51,141 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1359104465_0011_m_000000_0
2015-02-10 22:48:51,146 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,148 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:51,148 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@41c17a8c
2015-02-10 22:48:51,150 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:51,158 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:51,159 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,169 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:51,169 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 22:48:51,181 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:51,182 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1359104465_0011_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,183 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:51,183 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1359104465_0011_m_000000_0 is allowed to commit now
2015-02-10 22:48:51,184 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1359104465_0011_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local1359104465_0011_m_000000
2015-02-10 22:48:51,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:51,186 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1359104465_0011_m_000000_0' done.
2015-02-10 22:48:51,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1359104465_0011_m_000000_0
2015-02-10 22:48:51,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:51,206 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2015-02-10 22:48:51,241 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,243 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2015-02-10 22:48:51,266 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:51,267 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:51,278 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:51,281 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:51,282 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:51,289 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:51,289 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,290 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 22:48:51,290 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 22:48:51,290 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 22:48:51,290 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 22:48:51,291 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2015-02-10 22:48:51,292 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,294 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,321 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,342 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,359 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1277683109_0012
2015-02-10 22:48:51,386 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1277683109/.staging/job_local1277683109_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,387 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1277683109/.staging/job_local1277683109_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,440 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1277683109_0012/job_local1277683109_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,441 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1277683109_0012/job_local1277683109_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,442 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,442 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1277683109_0012
2015-02-10 22:48:51,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,442 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:51,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,445 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,445 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,447 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,449 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:51,449 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3358fd70
2015-02-10 22:48:51,450 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:51,451 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:51,516 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:51,516 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:51,516 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:51,516 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:51,516 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:51,518 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:51,518 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,528 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,528 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:51,529 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:51,531 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:51,531 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,531 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,532 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:51,532 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,533 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,534 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:51,534 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,534 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,535 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:51,536 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 497, max: 672
2015-02-10 22:48:51,536 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 73% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 22:48:51,538 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:51,538 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:51,538 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:51,539 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:51,539 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:51,543 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:51,546 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1277683109_0012_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:51,549 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1277683109_0012_m_000000_0' done.
2015-02-10 22:48:51,549 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:51,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:51,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1277683109_0012_r_000000_0
2015-02-10 22:48:51,553 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,554 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@402fbd59
2015-02-10 22:48:51,555 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:51,557 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1277683109_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:51,559 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local1277683109_0012_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:51,560 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1277683109_0012_m_000000_0
2015-02-10 22:48:51,560 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:51,560 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:51,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,563 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:51,566 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:51,566 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:51,568 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:51,569 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:51,569 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:51,570 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:51,570 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:51,571 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,572 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:51,572 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:51,583 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 22:48:51,584 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 22:48:51,597 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1277683109_0012_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:51,600 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:51,600 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1277683109_0012_r_000000_0 is allowed to commit now
2015-02-10 22:48:51,601 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1277683109_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local1277683109_0012_r_000000
2015-02-10 22:48:51,602 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:51,603 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1277683109_0012_r_000000_0' done.
2015-02-10 22:48:51,603 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1277683109_0012_r_000000_0
2015-02-10 22:48:51,603 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:51,626 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2015-02-10 22:48:51,672 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,704 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,706 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2015-02-10 22:48:51,728 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:51,729 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:51,737 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:51,739 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:51,740 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:51,745 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:51,746 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:51,746 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 22:48:51,746 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 22:48:51,747 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 22:48:51,747 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 22:48:51,748 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2015-02-10 22:48:51,749 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,750 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:51,775 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:51,796 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:51,812 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local971840396_0013
2015-02-10 22:48:51,830 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity971840396/.staging/job_local971840396_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,831 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity971840396/.staging/job_local971840396_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,881 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local971840396_0013/job_local971840396_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:51,883 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local971840396_0013/job_local971840396_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:51,883 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:51,884 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local971840396_0013
2015-02-10 22:48:51,884 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:51,884 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:51,884 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:51,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:51,887 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:51,890 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:51,892 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:51,892 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ec1e10c
2015-02-10 22:48:51,894 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:51,894 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,040 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,041 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,041 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,041 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,041 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,043 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,043 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,052 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,052 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:52,053 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,054 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,054 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,055 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,055 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,056 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,056 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,057 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,057 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 144, total: 599, max: 672
2015-02-10 22:48:52,058 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:52,059 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,059 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,059 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 22:48:52,059 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,062 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,064 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local971840396_0013_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,066 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,067 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local971840396_0013_m_000000_0' done.
2015-02-10 22:48:52,067 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:52,067 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,068 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,068 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local971840396_0013_r_000000_0
2015-02-10 22:48:52,069 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,070 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1a3ae8ac
2015-02-10 22:48:52,070 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,071 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local971840396_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,074 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local971840396_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 22:48:52,074 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local971840396_0013_m_000000_0
2015-02-10 22:48:52,074 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 22:48:52,075 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,076 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,077 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,079 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,079 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,080 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,081 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 22:48:52,081 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,081 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,081 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,082 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,082 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,083 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,091 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:52,091 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 22:48:52,102 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local971840396_0013_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,104 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,104 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local971840396_0013_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,106 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local971840396_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local971840396_0013_r_000000
2015-02-10 22:48:52,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,108 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local971840396_0013_r_000000_0' done.
2015-02-10 22:48:52,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local971840396_0013_r_000000_0
2015-02-10 22:48:52,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,126 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2015-02-10 22:48:52,159 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,194 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,196 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2015-02-10 22:48:52,218 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:52,218 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:52,227 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:52,229 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:52,230 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:52,235 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 22:48:52,235 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,235 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 22:48:52,236 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] parallel execution is enabled: false
2015-02-10 22:48:52,236 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting jobs: 1
2015-02-10 22:48:52,236 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [min] allocating threads: 1
2015-02-10 22:48:52,236 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2015-02-10 22:48:52,237 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,238 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,264 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,282 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:52,298 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local863072059_0014
2015-02-10 22:48:52,318 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity863072059/.staging/job_local863072059_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:52,319 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity863072059/.staging/job_local863072059_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:52,370 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local863072059_0014/job_local863072059_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:52,372 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local863072059_0014/job_local863072059_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:52,372 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:52,373 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local863072059_0014
2015-02-10 22:48:52,373 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:52,373 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 22:48:52,373 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:52,376 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:52,376 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,378 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,380 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:52,380 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14d6c410
2015-02-10 22:48:52,381 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:52,381 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,411 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,411 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,413 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,413 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,413 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,414 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,414 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,423 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,424 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:52,424 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,426 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,426 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,427 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,427 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,427 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,428 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,428 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,428 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 181, total: 689, max: 689
2015-02-10 22:48:52,430 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:52,430 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,430 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,430 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 22:48:52,430 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,434 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,436 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local863072059_0014_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,441 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,441 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local863072059_0014_m_000000_0' done.
2015-02-10 22:48:52,441 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local863072059_0014_r_000000_0
2015-02-10 22:48:52,444 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,444 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c4737a3
2015-02-10 22:48:52,445 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=505957568, maxSingleShuffleLimit=126489392, mergeThreshold=333932000, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,446 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local863072059_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,448 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local863072059_0014_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 22:48:52,449 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local863072059_0014_m_000000_0
2015-02-10 22:48:52,449 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 22:48:52,450 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,451 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,452 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,454 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,454 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,455 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,456 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 22:48:52,456 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,456 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,456 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 22:48:52,457 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,457 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,457 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,466 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 22:48:52,466 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 22:48:52,476 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local863072059_0014_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,478 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,478 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local863072059_0014_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,479 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local863072059_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local863072059_0014_r_000000
2015-02-10 22:48:52,480 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,480 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local863072059_0014_r_000000_0' done.
2015-02-10 22:48:52,480 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local863072059_0014_r_000000_0
2015-02-10 22:48:52,481 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,501 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2015-02-10 22:48:52,536 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,569 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,570 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2015-02-10 22:48:52,597 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:52,598 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:52,609 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:52,612 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:52,613 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:52,620 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 22:48:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 22:48:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] parallel execution is enabled: false
2015-02-10 22:48:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting jobs: 1
2015-02-10 22:48:52,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] allocating threads: 1
2015-02-10 22:48:52,622 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2015-02-10 22:48:52,623 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,624 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:52,649 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:52,674 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:52,697 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local965179374_0015
2015-02-10 22:48:52,722 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity965179374/.staging/job_local965179374_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:52,725 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity965179374/.staging/job_local965179374_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:52,794 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local965179374_0015/job_local965179374_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:52,797 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local965179374_0015/job_local965179374_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:52,798 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:52,798 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:52,799 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:52,798 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local965179374_0015
2015-02-10 22:48:52,799 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 22:48:52,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:52,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,804 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,807 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:52,807 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7d786789
2015-02-10 22:48:52,809 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:52,810 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:52,840 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:52,840 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:52,840 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:52,840 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:52,841 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:52,842 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:52,842 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,852 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:52,852 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:52,853 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:52,854 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:52,854 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,855 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:52,856 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,856 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:52,856 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,857 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:52,857 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 184, total: 692, max: 692
2015-02-10 22:48:52,858 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:52,859 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:52,859 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:52,859 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:52,859 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:52,862 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:52,864 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local965179374_0015_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:52,870 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local965179374_0015_m_000000_0' done.
2015-02-10 22:48:52,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,871 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:52,871 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:52,871 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local965179374_0015_r_000000_0
2015-02-10 22:48:52,873 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:52,874 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@65a64b69
2015-02-10 22:48:52,874 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=508434848, maxSingleShuffleLimit=127108712, mergeThreshold=335567008, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:52,876 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local965179374_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:52,878 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local965179374_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:52,879 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local965179374_0015_m_000000_0
2015-02-10 22:48:52,879 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:52,880 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:52,881 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,881 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:52,885 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,885 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:52,887 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:52,887 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:52,887 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:52,888 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:52,888 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:52,889 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,889 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:52,889 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:52,898 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:52,899 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 22:48:52,910 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local965179374_0015_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:52,912 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:52,912 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local965179374_0015_r_000000_0 is allowed to commit now
2015-02-10 22:48:52,913 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local965179374_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local965179374_0015_r_000000
2015-02-10 22:48:52,914 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:52,914 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local965179374_0015_r_000000_0' done.
2015-02-10 22:48:52,914 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local965179374_0015_r_000000_0
2015-02-10 22:48:52,915 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:52,940 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2015-02-10 22:48:52,971 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,005 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,007 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2015-02-10 22:48:53,031 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,032 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,044 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,048 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,048 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,054 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting
2015-02-10 22:48:53,054 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:53,054 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:53,054 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 22:48:53,054 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] parallel execution is enabled: false
2015-02-10 22:48:53,055 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting jobs: 1
2015-02-10 22:48:53,055 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] allocating threads: 1
2015-02-10 22:48:53,055 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2015-02-10 22:48:53,056 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,057 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,082 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,083 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,102 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:53,124 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1315961791_0016
2015-02-10 22:48:53,146 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1315961791/.staging/job_local1315961791_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,147 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1315961791/.staging/job_local1315961791_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,200 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1315961791_0016/job_local1315961791_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,201 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1315961791_0016/job_local1315961791_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,202 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,202 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,203 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local1315961791_0016
2015-02-10 22:48:53,203 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2015-02-10 22:48:53,203 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,205 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,206 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,207 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,208 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:53,209 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@73b829e7
2015-02-10 22:48:53,210 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:53,210 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:53,238 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:53,238 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:53,238 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:53,238 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:53,238 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:53,240 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,240 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,251 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:53,251 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,252 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:53,253 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:53,254 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,255 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:53,255 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,255 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:53,256 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 231, total: 739, max: 739
2015-02-10 22:48:53,256 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:53,256 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 230, total: 739, max: 739
2015-02-10 22:48:53,258 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:53,258 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:53,258 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:53,258 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:53,258 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:53,261 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:53,263 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,265 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:53,266 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_m_000000_0' done.
2015-02-10 22:48:53,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,266 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,267 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,269 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:53,269 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6cab6dcb
2015-02-10 22:48:53,270 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:53,270 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:53,351 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:53,352 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:53,352 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:53,352 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:53,352 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:53,353 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,353 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,365 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:53,365 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,366 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:53,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:53,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:53,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:53,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:53,373 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 129, total: 739, max: 739
2015-02-10 22:48:53,375 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:53,375 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:53,375 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:53,375 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:53,375 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:53,379 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:53,381 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:53,383 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:53,383 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_m_000001_0' done.
2015-02-10 22:48:53,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:53,384 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:53,385 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1315961791_0016_r_000000_0
2015-02-10 22:48:53,386 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,387 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7faea002
2015-02-10 22:48:53,389 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=542932992, maxSingleShuffleLimit=135733248, mergeThreshold=358335776, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:53,390 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1315961791_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:53,392 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1315961791_0016_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:53,393 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1315961791_0016_m_000001_0
2015-02-10 22:48:53,393 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:53,396 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local1315961791_0016_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:53,396 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1315961791_0016_m_000000_0
2015-02-10 22:48:53,397 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 22:48:53,397 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:53,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,398 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:53,401 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:53,401 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 22:48:53,403 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:53,403 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 22:48:53,403 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:53,404 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:53,404 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 22:48:53,405 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,405 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:53,405 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,416 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 22:48:53,417 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 22:48:53,428 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1315961791_0016_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,430 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:53,430 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1315961791_0016_r_000000_0 is allowed to commit now
2015-02-10 22:48:53,431 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1315961791_0016_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local1315961791_0016_r_000000
2015-02-10 22:48:53,432 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:53,432 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1315961791_0016_r_000000_0' done.
2015-02-10 22:48:53,432 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1315961791_0016_r_000000_0
2015-02-10 22:48:53,433 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:53,453 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2015-02-10 22:48:53,487 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,525 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,527 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2015-02-10 22:48:53,553 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,554 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,562 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,574 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,574 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,580 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:53,580 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,580 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 22:48:53,580 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:48:53,580 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:48:53,581 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:48:53,581 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2015-02-10 22:48:53,582 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,583 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,606 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,625 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:53,641 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local60824353_0017
2015-02-10 22:48:53,660 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity60824353/.staging/job_local60824353_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,661 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity60824353/.staging/job_local60824353_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,707 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local60824353_0017/job_local60824353_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,708 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local60824353_0017/job_local60824353_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,709 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,709 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local60824353_0017
2015-02-10 22:48:53,709 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,710 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:53,710 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,712 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,713 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local60824353_0017_m_000000_0
2015-02-10 22:48:53,715 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,716 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:53,717 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@463b7f4e
2015-02-10 22:48:53,718 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:53,726 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,726 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,732 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,732 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 22:48:53,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:53,736 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local60824353_0017_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:53,737 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:53,737 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local60824353_0017_m_000000_0 is allowed to commit now
2015-02-10 22:48:53,738 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local60824353_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local60824353_0017_m_000000
2015-02-10 22:48:53,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:53,739 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local60824353_0017_m_000000_0' done.
2015-02-10 22:48:53,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local60824353_0017_m_000000_0
2015-02-10 22:48:53,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:53,754 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2015-02-10 22:48:53,785 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,787 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2015-02-10 22:48:53,807 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:53,807 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:53,814 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:53,816 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:53,817 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:53,820 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:53,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 22:48:53,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:48:53,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:48:53,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:48:53,822 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2015-02-10 22:48:53,823 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,824 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:53,847 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:53,866 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:53,882 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local541929438_0018
2015-02-10 22:48:53,900 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity541929438/.staging/job_local541929438_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,902 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity541929438/.staging/job_local541929438_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,949 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local541929438_0018/job_local541929438_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:53,950 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local541929438_0018/job_local541929438_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:53,951 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:53,951 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local541929438_0018
2015-02-10 22:48:53,951 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:53,951 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:53,952 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:53,954 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:53,954 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local541929438_0018_m_000000_0
2015-02-10 22:48:53,956 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:53,957 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:53,958 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34ac3c11
2015-02-10 22:48:53,959 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:53,966 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:53,966 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:53,972 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:53,972 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 22:48:53,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:54,027 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local541929438_0018_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,029 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:54,029 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local541929438_0018_m_000000_0 is allowed to commit now
2015-02-10 22:48:54,030 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local541929438_0018_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local541929438_0018_m_000000
2015-02-10 22:48:54,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:54,031 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local541929438_0018_m_000000_0' done.
2015-02-10 22:48:54,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local541929438_0018_m_000000_0
2015-02-10 22:48:54,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,046 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2015-02-10 22:48:54,078 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,080 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2015-02-10 22:48:54,105 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:54,106 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:54,121 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,124 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,125 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:54,132 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 22:48:54,133 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 22:48:54,134 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2015-02-10 22:48:54,135 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,136 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,160 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,161 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,181 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:54,197 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1534238509_0019
2015-02-10 22:48:54,215 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1534238509/.staging/job_local1534238509_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:54,217 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1534238509/.staging/job_local1534238509_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:54,265 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1534238509_0019/job_local1534238509_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:54,267 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1534238509_0019/job_local1534238509_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:54,268 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:54,269 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:54,269 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:54,269 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1534238509_0019
2015-02-10 22:48:54,270 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:54,272 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:54,272 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,273 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,274 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:54,275 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2b76fbc2
2015-02-10 22:48:54,276 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,276 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,303 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,303 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,303 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,303 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,304 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,319 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,319 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,319 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,321 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,321 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,322 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,322 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,323 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,323 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,324 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,324 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:54,326 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:54,326 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,326 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,326 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:54,326 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,329 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,331 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:54,334 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_m_000000_0' done.
2015-02-10 22:48:54,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,336 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,337 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:48:54,338 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3aa8c31b
2015-02-10 22:48:54,339 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,339 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,360 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,361 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,361 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,361 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,362 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,376 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:48:54,377 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,377 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,382 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,382 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,384 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,384 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,385 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,385 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,386 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,386 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 136, total: 750, max: 750
2015-02-10 22:48:54,388 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:54,388 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,388 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,389 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 22:48:54,389 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,392 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,395 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:54,397 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:48:54,397 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_m_000001_0' done.
2015-02-10 22:48:54,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:54,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1534238509_0019_r_000000_0
2015-02-10 22:48:54,401 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,402 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3b5b0734
2015-02-10 22:48:54,402 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=550869376, maxSingleShuffleLimit=137717344, mergeThreshold=363573792, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:54,403 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1534238509_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:54,407 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1534238509_0019_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:54,408 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1534238509_0019_m_000001_0
2015-02-10 22:48:54,408 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 22:48:54,411 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1534238509_0019_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 22:48:54,411 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1534238509_0019_m_000000_0
2015-02-10 22:48:54,411 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 22:48:54,412 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:54,413 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,413 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:54,416 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:54,417 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 22:48:54,419 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:54,420 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 22:48:54,420 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:54,420 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,421 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 22:48:54,422 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,423 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:54,423 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,436 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:54,436 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 22:48:54,450 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1534238509_0019_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:54,452 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1534238509_0019_r_000000_0 is allowed to commit now
2015-02-10 22:48:54,453 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1534238509_0019_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local1534238509_0019_r_000000
2015-02-10 22:48:54,455 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:54,455 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1534238509_0019_r_000000_0' done.
2015-02-10 22:48:54,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1534238509_0019_r_000000_0
2015-02-10 22:48:54,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:54,477 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2015-02-10 22:48:54,511 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,549 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,551 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2015-02-10 22:48:54,579 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:54,580 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:54,607 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,609 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,616 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:54,622 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting
2015-02-10 22:48:54,623 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,623 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 22:48:54,623 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] parallel execution is enabled: false
2015-02-10 22:48:54,624 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting jobs: 2
2015-02-10 22:48:54,624 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [second] allocating threads: 1
2015-02-10 22:48:54,625 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2015-02-10 22:48:54,626 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,627 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,662 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,681 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:54,696 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2033889522_0020
2015-02-10 22:48:54,715 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2033889522/.staging/job_local2033889522_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:54,716 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2033889522/.staging/job_local2033889522_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:54,764 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2033889522_0020/job_local2033889522_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:54,765 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2033889522_0020/job_local2033889522_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:54,766 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:54,766 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:54,767 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:54,767 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local2033889522_0020
2015-02-10 22:48:54,767 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 22:48:54,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:54,769 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,771 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,772 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:54,773 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1330c35c
2015-02-10 22:48:54,774 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:54,774 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:54,794 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:54,794 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:54,794 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:54,794 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:54,795 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:54,796 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:54,797 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,807 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:54,808 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2015-02-10 22:48:54,808 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:54,812 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:54,813 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,814 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:54,814 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:54,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:54,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 206, total: 750, max: 750
2015-02-10 22:48:54,817 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:54,817 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:54,817 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:54,817 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:54,817 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:54,821 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:54,823 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2033889522_0020_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,825 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:54,825 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2033889522_0020_m_000000_0' done.
2015-02-10 22:48:54,825 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,825 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:54,826 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:54,826 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2033889522_0020_r_000000_0
2015-02-10 22:48:54,829 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:54,830 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@17ccad9c
2015-02-10 22:48:54,832 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:54,832 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2033889522_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:54,834 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local2033889522_0020_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:54,835 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local2033889522_0020_m_000000_0
2015-02-10 22:48:54,835 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:54,837 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:54,838 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,838 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:54,840 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,840 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:54,842 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:54,842 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:54,842 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:54,843 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:54,843 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:54,844 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,845 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:54,845 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:54,853 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2015-02-10 22:48:54,854 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][7042273824/first/]
2015-02-10 22:48:54,863 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2033889522_0020_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:54,864 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:54,865 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2033889522_0020_r_000000_0 is allowed to commit now
2015-02-10 22:48:54,865 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2033889522_0020_r_000000_0' to file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/_temporary/0/task_local2033889522_0020_r_000000
2015-02-10 22:48:54,866 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:54,866 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2033889522_0020_r_000000_0' done.
2015-02-10 22:48:54,867 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2033889522_0020_r_000000_0
2015-02-10 22:48:54,867 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:54,885 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2015-02-10 22:48:54,886 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,887 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:54,923 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:54,942 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:54,952 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:54,952 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:54,958 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1981796583_0021
2015-02-10 22:48:54,975 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1981796583/.staging/job_local1981796583_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:54,976 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1981796583/.staging/job_local1981796583_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,024 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981796583_0021/job_local1981796583_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:55,025 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1981796583_0021/job_local1981796583_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,025 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,026 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1981796583_0021
2015-02-10 22:48:55,026 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,026 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 22:48:55,026 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,029 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,029 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,031 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,032 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/part-00000
2015-02-10 22:48:55,032 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1367dca
2015-02-10 22:48:55,034 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,034 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,061 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,062 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,062 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,062 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,062 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,063 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,063 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,074 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][7042273824/first/]
2015-02-10 22:48:55,074 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2015-02-10 22:48:55,075 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,076 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,076 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:55,077 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,077 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 22:48:55,079 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:55,079 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,079 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,079 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:55,079 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:55,082 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,083 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1981796583_0021_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,085 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/7042273824_first_F9B9801D789A48268AB65870D7CE7FC8/part-00000:0+160
2015-02-10 22:48:55,086 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981796583_0021_m_000000_0' done.
2015-02-10 22:48:55,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,086 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,087 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1981796583_0021_r_000000_0
2015-02-10 22:48:55,088 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,088 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3f4daa36
2015-02-10 22:48:55,089 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,090 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1981796583_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,092 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local1981796583_0021_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:55,092 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1981796583_0021_m_000000_0
2015-02-10 22:48:55,092 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:55,093 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,095 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,095 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,097 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,098 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:55,099 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,100 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:55,100 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,101 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,101 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:55,102 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,102 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,103 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,113 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2015-02-10 22:48:55,113 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 22:48:55,124 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1981796583_0021_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,125 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,125 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1981796583_0021_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,126 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1981796583_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local1981796583_0021_r_000000
2015-02-10 22:48:55,127 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,128 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1981796583_0021_r_000000_0' done.
2015-02-10 22:48:55,128 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1981796583_0021_r_000000_0
2015-02-10 22:48:55,128 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,153 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2015-02-10 22:48:55,182 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,213 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,215 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2015-02-10 22:48:55,235 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,236 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,243 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,245 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,245 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,250 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 22:48:55,250 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,250 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 22:48:55,251 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] parallel execution is enabled: false
2015-02-10 22:48:55,251 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting jobs: 1
2015-02-10 22:48:55,251 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] allocating threads: 1
2015-02-10 22:48:55,251 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2015-02-10 22:48:55,252 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,253 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,279 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,301 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:55,317 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local273492952_0022
2015-02-10 22:48:55,334 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity273492952/.staging/job_local273492952_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:55,336 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity273492952/.staging/job_local273492952_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,382 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local273492952_0022/job_local273492952_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:55,384 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local273492952_0022/job_local273492952_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,384 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,385 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,385 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,385 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local273492952_0022
2015-02-10 22:48:55,386 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 22:48:55,388 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,388 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,390 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,392 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:55,392 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@62892cc5
2015-02-10 22:48:55,394 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,395 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,423 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,423 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,423 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,423 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,424 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,425 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,425 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,434 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,435 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:55,436 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,437 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,438 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,439 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,439 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,440 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:55,440 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,441 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:55,441 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,442 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:55,443 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,443 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,443 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:55,444 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:55,447 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,449 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local273492952_0022_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,451 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:55,451 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local273492952_0022_m_000000_0' done.
2015-02-10 22:48:55,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,452 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,453 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local273492952_0022_r_000000_0
2015-02-10 22:48:55,456 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,456 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4005f23d
2015-02-10 22:48:55,457 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551328128, maxSingleShuffleLimit=137832032, mergeThreshold=363876576, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,457 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local273492952_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,459 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local273492952_0022_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:55,460 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local273492952_0022_m_000000_0
2015-02-10 22:48:55,460 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:55,461 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,462 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,462 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,464 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,464 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:55,466 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,466 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 22:48:55,466 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,466 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,467 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:55,467 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,467 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,468 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,474 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:55,474 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 22:48:55,485 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local273492952_0022_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,486 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,486 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local273492952_0022_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,487 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local273492952_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local273492952_0022_r_000000
2015-02-10 22:48:55,488 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,488 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local273492952_0022_r_000000_0' done.
2015-02-10 22:48:55,489 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local273492952_0022_r_000000_0
2015-02-10 22:48:55,489 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,512 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2015-02-10 22:48:55,543 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,577 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,580 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2015-02-10 22:48:55,602 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,602 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,609 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,612 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,613 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,617 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 22:48:55,618 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,618 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 22:48:55,618 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] parallel execution is enabled: false
2015-02-10 22:48:55,618 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting jobs: 1
2015-02-10 22:48:55,618 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [max] allocating threads: 1
2015-02-10 22:48:55,619 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...splatform/maxbynullsafety
2015-02-10 22:48:55,620 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,621 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,645 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,664 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:55,679 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1009360269_0023
2015-02-10 22:48:55,696 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1009360269/.staging/job_local1009360269_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:55,697 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1009360269/.staging/job_local1009360269_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,746 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1009360269_0023/job_local1009360269_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:55,747 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1009360269_0023/job_local1009360269_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:55,748 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:55,748 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1009360269_0023
2015-02-10 22:48:55,748 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:55,749 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:55,748 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 22:48:55,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:55,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,759 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,761 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:55,761 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7d4720ef
2015-02-10 22:48:55,762 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:55,762 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:55,790 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:55,791 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:55,791 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:55,791 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:55,791 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:55,793 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:55,793 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,800 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,801 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:55,801 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:55,803 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:55,803 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,804 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:55,805 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,806 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:55,806 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,807 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:55,807 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 751, max: 751
2015-02-10 22:48:55,808 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:55,809 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:55,809 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:55,809 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 98; bufvoid = 104857600
2015-02-10 22:48:55,810 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:55,813 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:55,815 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1009360269_0023_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,817 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:55,817 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1009360269_0023_m_000000_0' done.
2015-02-10 22:48:55,818 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,819 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:55,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:55,820 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1009360269_0023_r_000000_0
2015-02-10 22:48:55,821 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:55,822 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@47a77e1e
2015-02-10 22:48:55,822 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551374016, maxSingleShuffleLimit=137843504, mergeThreshold=363906880, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:55,823 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1009360269_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:55,824 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local1009360269_0023_m_000000_0 decomp: 118 len: 122 to MEMORY
2015-02-10 22:48:55,825 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 118 bytes from map-output for attempt_local1009360269_0023_m_000000_0
2015-02-10 22:48:55,825 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 118, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->118
2015-02-10 22:48:55,826 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:55,827 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,828 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:55,829 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,830 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 22:48:55,831 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 118 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:55,832 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 122 bytes from disk
2015-02-10 22:48:55,832 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:55,832 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:55,832 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 22:48:55,833 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,833 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:55,833 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:55,841 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 22:48:55,841 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 22:48:55,852 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1009360269_0023_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:55,854 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:55,854 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1009360269_0023_r_000000_0 is allowed to commit now
2015-02-10 22:48:55,855 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1009360269_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary/0/task_local1009360269_0023_r_000000
2015-02-10 22:48:55,856 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:55,856 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1009360269_0023_r_000000_0' done.
2015-02-10 22:48:55,856 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1009360269_0023_r_000000_0
2015-02-10 22:48:55,856 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:55,880 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary
2015-02-10 22:48:55,910 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,940 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:55,942 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety
2015-02-10 22:48:55,963 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:55,963 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:55,970 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:55,973 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:55,974 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:55,980 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 22:48:55,980 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:55,980 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 22:48:55,980 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] parallel execution is enabled: false
2015-02-10 22:48:55,982 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting jobs: 1
2015-02-10 22:48:55,982 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] allocating threads: 1
2015-02-10 22:48:55,983 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2015-02-10 22:48:55,984 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:55,985 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,008 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,026 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,041 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1796135208_0024
2015-02-10 22:48:56,058 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1796135208/.staging/job_local1796135208_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,059 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1796135208/.staging/job_local1796135208_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,108 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1796135208_0024/job_local1796135208_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,109 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1796135208_0024/job_local1796135208_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,109 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,110 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1796135208_0024
2015-02-10 22:48:56,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,110 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 22:48:56,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,117 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,118 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,118 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1d272e52
2015-02-10 22:48:56,120 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,120 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,149 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,149 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,149 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,149 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,149 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,151 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,151 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:56,161 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,170 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:56,170 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,171 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:56,171 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,172 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:56,172 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,173 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:56,173 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:56,174 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,174 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,175 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2015-02-10 22:48:56,175 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:56,178 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,180 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1796135208_0024_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,182 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,182 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1796135208_0024_m_000000_0' done.
2015-02-10 22:48:56,182 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,183 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,183 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,184 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1796135208_0024_r_000000_0
2015-02-10 22:48:56,185 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,186 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@291972f8
2015-02-10 22:48:56,186 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551603392, maxSingleShuffleLimit=137900848, mergeThreshold=364058240, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,187 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1796135208_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,190 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local1796135208_0024_m_000000_0 decomp: 116 len: 120 to MEMORY
2015-02-10 22:48:56,190 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local1796135208_0024_m_000000_0
2015-02-10 22:48:56,190 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2015-02-10 22:48:56,191 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,192 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,193 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,195 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,195 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 22:48:56,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 120 bytes from disk
2015-02-10 22:48:56,197 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,197 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,197 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 22:48:56,198 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,198 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,199 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,204 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 22:48:56,205 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 22:48:56,215 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1796135208_0024_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,216 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,216 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1796135208_0024_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,217 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1796135208_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local1796135208_0024_r_000000
2015-02-10 22:48:56,218 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,218 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1796135208_0024_r_000000_0' done.
2015-02-10 22:48:56,218 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1796135208_0024_r_000000_0
2015-02-10 22:48:56,219 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,236 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2015-02-10 22:48:56,267 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,298 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,300 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2015-02-10 22:48:56,321 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:56,322 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:56,328 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:56,330 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:56,331 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:56,335 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:48:56,335 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,336 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 22:48:56,336 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 22:48:56,336 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 22:48:56,336 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 22:48:56,337 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2015-02-10 22:48:56,337 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,338 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,363 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,381 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,398 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1793719809_0025
2015-02-10 22:48:56,416 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1793719809/.staging/job_local1793719809_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,417 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1793719809/.staging/job_local1793719809_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,466 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1793719809_0025/job_local1793719809_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,467 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1793719809_0025/job_local1793719809_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,467 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,468 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1793719809_0025
2015-02-10 22:48:56,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,468 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:48:56,471 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,471 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,472 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,474 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,474 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@30e6ace7
2015-02-10 22:48:56,475 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,475 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,503 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,504 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,504 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,504 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,504 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,505 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,505 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,513 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,513 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:48:56,514 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,517 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:56,517 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,517 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,518 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2015-02-10 22:48:56,519 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 22:48:56,523 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,525 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1793719809_0025_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,528 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,528 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1793719809_0025_m_000000_0' done.
2015-02-10 22:48:56,528 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1793719809_0025_r_000000_0
2015-02-10 22:48:56,532 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,533 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@65e7c41f
2015-02-10 22:48:56,533 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551649280, maxSingleShuffleLimit=137912320, mergeThreshold=364088544, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,534 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1793719809_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,536 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local1793719809_0025_m_000000_0 decomp: 8 len: 12 to MEMORY
2015-02-10 22:48:56,536 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local1793719809_0025_m_000000_0
2015-02-10 22:48:56,536 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2015-02-10 22:48:56,538 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,539 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,540 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,542 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,542 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 22:48:56,543 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,544 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 12 bytes from disk
2015-02-10 22:48:56,544 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,544 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,544 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 22:48:56,545 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,545 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,546 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,553 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:48:56,553 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 22:48:56,562 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1793719809_0025_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,564 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1793719809_0025_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,564 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1793719809_0025_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local1793719809_0025_r_000000
2015-02-10 22:48:56,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,566 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1793719809_0025_r_000000_0' done.
2015-02-10 22:48:56,566 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1793719809_0025_r_000000_0
2015-02-10 22:48:56,566 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,581 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2015-02-10 22:48:56,615 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,651 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,653 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2015-02-10 22:48:56,677 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:56,678 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:56,687 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:56,688 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:56,689 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:56,694 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 22:48:56,695 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,695 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 22:48:56,696 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] parallel execution is enabled: false
2015-02-10 22:48:56,696 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting jobs: 1
2015-02-10 22:48:56,696 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [name] allocating threads: 1
2015-02-10 22:48:56,697 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2015-02-10 22:48:56,698 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,699 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:56,723 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:56,744 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:56,759 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local105142053_0026
2015-02-10 22:48:56,777 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity105142053/.staging/job_local105142053_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,778 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity105142053/.staging/job_local105142053_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,824 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local105142053_0026/job_local105142053_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:56,826 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local105142053_0026/job_local105142053_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:56,826 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:56,827 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:56,827 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:56,827 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local105142053_0026
2015-02-10 22:48:56,828 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 22:48:56,830 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:56,830 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,831 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,832 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:56,833 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1ea44130
2015-02-10 22:48:56,834 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:56,834 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:56,862 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:56,862 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:56,862 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:56,862 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:56,862 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:56,864 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:56,864 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,874 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:56,874 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:56,874 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:56,876 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:56,876 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,878 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:56,878 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,879 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:56,879 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,879 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:56,880 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 751, max: 751
2015-02-10 22:48:56,881 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:56,881 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:56,881 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:56,881 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2015-02-10 22:48:56,882 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:56,885 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:56,886 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local105142053_0026_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,888 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:56,888 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local105142053_0026_m_000000_0' done.
2015-02-10 22:48:56,889 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,889 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:56,889 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:56,889 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local105142053_0026_r_000000_0
2015-02-10 22:48:56,892 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:56,892 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3997ebf6
2015-02-10 22:48:56,893 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551878656, maxSingleShuffleLimit=137969664, mergeThreshold=364239936, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:56,894 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local105142053_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:56,896 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local105142053_0026_m_000000_0 decomp: 452 len: 456 to MEMORY
2015-02-10 22:48:56,897 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local105142053_0026_m_000000_0
2015-02-10 22:48:56,897 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2015-02-10 22:48:56,898 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:56,899 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,899 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:56,901 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,901 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 22:48:56,903 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:56,903 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 456 bytes from disk
2015-02-10 22:48:56,903 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:56,903 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:56,904 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 22:48:56,904 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,905 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:56,905 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:56,914 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 22:48:56,914 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 22:48:56,926 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local105142053_0026_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:56,927 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:56,927 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local105142053_0026_r_000000_0 is allowed to commit now
2015-02-10 22:48:56,928 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local105142053_0026_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local105142053_0026_r_000000
2015-02-10 22:48:56,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:56,929 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local105142053_0026_r_000000_0' done.
2015-02-10 22:48:56,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local105142053_0026_r_000000_0
2015-02-10 22:48:56,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:56,950 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2015-02-10 22:48:56,979 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,013 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,014 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2015-02-10 22:48:57,033 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,033 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,038 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,040 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,040 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,045 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:57,045 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,045 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 22:48:57,045 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:48:57,046 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:48:57,046 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:48:57,047 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2015-02-10 22:48:57,047 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,049 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,075 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,094 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,115 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1476339018_0027
2015-02-10 22:48:57,134 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1476339018/.staging/job_local1476339018_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,136 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1476339018/.staging/job_local1476339018_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,188 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1476339018_0027/job_local1476339018_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,189 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1476339018_0027/job_local1476339018_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,190 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,191 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1476339018_0027
2015-02-10 22:48:57,191 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,191 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:57,191 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,194 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,194 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1476339018_0027_m_000000_0
2015-02-10 22:48:57,195 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,197 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:57,198 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@17a220d7
2015-02-10 22:48:57,199 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:57,207 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,207 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,212 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,212 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 22:48:57,215 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:57,215 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1476339018_0027_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,217 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:57,217 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1476339018_0027_m_000000_0 is allowed to commit now
2015-02-10 22:48:57,218 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1476339018_0027_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local1476339018_0027_m_000000
2015-02-10 22:48:57,219 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:57,219 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1476339018_0027_m_000000_0' done.
2015-02-10 22:48:57,219 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1476339018_0027_m_000000_0
2015-02-10 22:48:57,219 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,238 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2015-02-10 22:48:57,270 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,272 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2015-02-10 22:48:57,292 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,293 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,299 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,301 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,302 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,305 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:48:57,306 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,306 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 22:48:57,306 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:48:57,306 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:48:57,306 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:48:57,307 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2015-02-10 22:48:57,308 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,309 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,335 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,353 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,369 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1246039426_0028
2015-02-10 22:48:57,386 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1246039426/.staging/job_local1246039426_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,388 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1246039426/.staging/job_local1246039426_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,439 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1246039426_0028/job_local1246039426_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,441 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1246039426_0028/job_local1246039426_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,441 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,442 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1246039426_0028
2015-02-10 22:48:57,442 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:48:57,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,446 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,446 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1246039426_0028_m_000000_0
2015-02-10 22:48:57,448 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,449 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:57,450 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@bc0fc25
2015-02-10 22:48:57,451 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:57,459 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,459 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,463 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,463 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 22:48:57,466 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:57,466 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1246039426_0028_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,468 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:57,468 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1246039426_0028_m_000000_0 is allowed to commit now
2015-02-10 22:48:57,468 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1246039426_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local1246039426_0028_m_000000
2015-02-10 22:48:57,469 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:57,470 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1246039426_0028_m_000000_0' done.
2015-02-10 22:48:57,470 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1246039426_0028_m_000000_0
2015-02-10 22:48:57,470 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,486 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2015-02-10 22:48:57,518 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,521 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2015-02-10 22:48:57,543 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,544 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,551 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,552 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,553 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,559 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 22:48:57,559 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:57,559 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 22:48:57,560 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] parallel execution is enabled: false
2015-02-10 22:48:57,560 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting jobs: 1
2015-02-10 22:48:57,560 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [average] allocating threads: 1
2015-02-10 22:48:57,561 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2015-02-10 22:48:57,562 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,563 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,587 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,605 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,630 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local353443111_0029
2015-02-10 22:48:57,650 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity353443111/.staging/job_local353443111_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,651 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity353443111/.staging/job_local353443111_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,699 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local353443111_0029/job_local353443111_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,700 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local353443111_0029/job_local353443111_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:57,701 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:57,702 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local353443111_0029
2015-02-10 22:48:57,702 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:57,702 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 22:48:57,702 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:57,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:57,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,706 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,708 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:57,708 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5b606e1f
2015-02-10 22:48:57,709 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:57,709 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:57,729 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:57,729 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:57,730 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:57,730 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:57,730 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:57,731 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:57,731 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,739 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:57,739 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:57,739 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:57,743 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:57,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:57,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:57,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:57,747 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 221, total: 751, max: 751
2015-02-10 22:48:57,748 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:57,748 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:57,748 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:57,748 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 22:48:57,748 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:57,751 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:57,753 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local353443111_0029_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,755 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:57,755 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local353443111_0029_m_000000_0' done.
2015-02-10 22:48:57,755 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:57,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:57,756 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local353443111_0029_r_000000_0
2015-02-10 22:48:57,758 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:57,758 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1bb66a6b
2015-02-10 22:48:57,759 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551924544, maxSingleShuffleLimit=137981136, mergeThreshold=364270208, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:57,759 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local353443111_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:57,763 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local353443111_0029_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 22:48:57,763 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local353443111_0029_m_000000_0
2015-02-10 22:48:57,764 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 22:48:57,765 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:57,765 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,766 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:57,768 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:57,768 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:57,769 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:57,769 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 22:48:57,770 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:57,770 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:57,770 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 22:48:57,771 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,771 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:57,771 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:57,778 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 22:48:57,778 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 22:48:57,788 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local353443111_0029_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:57,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:57,790 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local353443111_0029_r_000000_0 is allowed to commit now
2015-02-10 22:48:57,790 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local353443111_0029_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local353443111_0029_r_000000
2015-02-10 22:48:57,792 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:57,792 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local353443111_0029_r_000000_0' done.
2015-02-10 22:48:57,792 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local353443111_0029_r_000000_0
2015-02-10 22:48:57,792 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:57,809 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2015-02-10 22:48:57,839 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,869 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,871 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2015-02-10 22:48:57,890 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:57,891 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:57,896 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:57,898 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:57,899 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:57,902 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 22:48:57,903 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:57,903 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 22:48:57,903 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] parallel execution is enabled: false
2015-02-10 22:48:57,903 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting jobs: 1
2015-02-10 22:48:57,903 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] allocating threads: 1
2015-02-10 22:48:57,904 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2015-02-10 22:48:57,904 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,905 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:57,928 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:57,946 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:57,961 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2036586552_0030
2015-02-10 22:48:57,978 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2036586552/.staging/job_local2036586552_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:57,979 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity2036586552/.staging/job_local2036586552_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,026 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2036586552_0030/job_local2036586552_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,027 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2036586552_0030/job_local2036586552_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,028 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,028 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local2036586552_0030
2015-02-10 22:48:58,028 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 22:48:58,030 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,030 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,034 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,034 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2036586552_0030_m_000000_0
2015-02-10 22:48:58,035 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,036 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt
2015-02-10 22:48:58,037 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@77dbc064
2015-02-10 22:48:58,038 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 22:48:58,044 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,044 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,048 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt"]
2015-02-10 22:48:58,049 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 22:48:58,051 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:58,051 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local2036586552_0030_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,052 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:58,053 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local2036586552_0030_m_000000_0 is allowed to commit now
2015-02-10 22:48:58,053 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2036586552_0030_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local2036586552_0030_m_000000
2015-02-10 22:48:58,054 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lower.txt:0+19
2015-02-10 22:48:58,054 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2036586552_0030_m_000000_0' done.
2015-02-10 22:48:58,054 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2036586552_0030_m_000000_0
2015-02-10 22:48:58,055 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,071 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2015-02-10 22:48:58,101 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,103 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2015-02-10 22:48:58,122 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:58,122 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:58,128 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,130 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,130 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:58,134 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 22:48:58,135 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 22:48:58,135 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 22:48:58,135 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] parallel execution is enabled: false
2015-02-10 22:48:58,135 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting jobs: 1
2015-02-10 22:48:58,135 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [first] allocating threads: 1
2015-02-10 22:48:58,136 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...tnfieldswithoutcomparator
2015-02-10 22:48:58,137 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,138 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,162 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,181 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,197 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1994969962_0031
2015-02-10 22:48:58,215 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1994969962/.staging/job_local1994969962_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,216 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1994969962/.staging/job_local1994969962_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,267 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1994969962_0031/job_local1994969962_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,268 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1994969962_0031/job_local1994969962_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,269 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,269 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1994969962_0031
2015-02-10 22:48:58,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,270 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 22:48:58,270 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,273 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,273 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,275 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,276 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt
2015-02-10 22:48:58,277 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@504be958
2015-02-10 22:48:58,278 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,278 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,311 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,311 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,311 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,311 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,311 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,313 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,313 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,318 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 22:48:58,318 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:58,319 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,321 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,321 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,323 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,323 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,325 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:58,325 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,325 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,325 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 22:48:58,325 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:58,328 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:58,329 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1994969962_0031_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs+rhs-cross-rev.txt:0+221
2015-02-10 22:48:58,331 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1994969962_0031_m_000000_0' done.
2015-02-10 22:48:58,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,332 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,332 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:58,332 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1994969962_0031_r_000000_0
2015-02-10 22:48:58,334 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,334 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3aa7ff2e
2015-02-10 22:48:58,335 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552153920, maxSingleShuffleLimit=138038480, mergeThreshold=364421600, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:58,335 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1994969962_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:58,337 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local1994969962_0031_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 22:48:58,338 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1994969962_0031_m_000000_0
2015-02-10 22:48:58,338 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 22:48:58,340 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:58,341 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,341 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:58,343 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,343 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 22:48:58,344 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 117 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:58,345 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 121 bytes from disk
2015-02-10 22:48:58,345 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:58,345 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,345 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 22:48:58,346 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,346 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:58,346 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,351 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 22:48:58,351 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 22:48:58,360 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1994969962_0031_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,362 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,362 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1994969962_0031_r_000000_0 is allowed to commit now
2015-02-10 22:48:58,363 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1994969962_0031_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary/0/task_local1994969962_0031_r_000000
2015-02-10 22:48:58,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:58,364 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1994969962_0031_r_000000_0' done.
2015-02-10 22:48:58,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1994969962_0031_r_000000_0
2015-02-10 22:48:58,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:58,386 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary
2015-02-10 22:48:58,415 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,417 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator
2015-02-10 22:48:58,442 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:48:58,442 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:48:58,469 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,471 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,493 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:48:58,498 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting
2015-02-10 22:48:58,498 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:58,499 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 22:48:58,499 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] parallel execution is enabled: false
2015-02-10 22:48:58,499 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting jobs: 5
2015-02-10 22:48:58,499 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] allocating threads: 1
2015-02-10 22:48:58,500 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2015-02-10 22:48:58,501 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,503 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,527 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,545 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,562 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1491073616_0032
2015-02-10 22:48:58,580 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1491073616/.staging/job_local1491073616_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,581 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1491073616/.staging/job_local1491073616_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,632 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491073616_0032/job_local1491073616_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,634 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491073616_0032/job_local1491073616_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,634 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,635 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1491073616_0032
2015-02-10 22:48:58,635 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:58,635 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,636 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,639 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,641 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,643 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:48:58,643 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@586ca3ef
2015-02-10 22:48:58,645 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,645 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,680 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,680 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,680 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,680 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,680 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,682 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,682 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,688 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:48:58,688 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2015-02-10 22:48:58,689 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,690 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,691 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,692 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,692 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 22:48:58,693 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,693 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 22:48:58,694 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,695 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:58,695 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,695 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,695 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 22:48:58,696 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 22:48:58,699 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:58,701 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1491073616_0032_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:48:58,705 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491073616_0032_m_000000_0' done.
2015-02-10 22:48:58,705 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,706 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:58,707 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:58,707 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1491073616_0032_r_000000_0
2015-02-10 22:48:58,708 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,709 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@687090a8
2015-02-10 22:48:58,709 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552383296, maxSingleShuffleLimit=138095824, mergeThreshold=364572992, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:58,710 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1491073616_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:58,713 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local1491073616_0032_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 22:48:58,713 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1491073616_0032_m_000000_0
2015-02-10 22:48:58,713 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 22:48:58,714 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:58,715 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,715 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:58,717 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,717 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:58,718 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:58,719 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 22:48:58,719 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:58,719 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:58,720 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 22:48:58,721 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,721 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:58,721 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,727 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2015-02-10 22:48:58,727 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:58,737 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1491073616_0032_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:58,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:58,739 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1491073616_0032_r_000000_0 is allowed to commit now
2015-02-10 22:48:58,739 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1491073616_0032_r_000000_0' to file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/_temporary/0/task_local1491073616_0032_r_000000
2015-02-10 22:48:58,741 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:58,741 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491073616_0032_r_000000_0' done.
2015-02-10 22:48:58,741 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1491073616_0032_r_000000_0
2015-02-10 22:48:58,741 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:58,764 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2015-02-10 22:48:58,764 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,765 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:58,802 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:58,821 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:58,833 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:58,834 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:58,841 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1104083515_0033
2015-02-10 22:48:58,862 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1104083515/.staging/job_local1104083515_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,863 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1104083515/.staging/job_local1104083515_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,913 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1104083515_0033/job_local1104083515_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:58,915 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1104083515_0033/job_local1104083515_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:58,915 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:58,916 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1104083515_0033
2015-02-10 22:48:58,916 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:58,916 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:58,918 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:58,921 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:58,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:58,924 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:58,926 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:58,927 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4b5bc191
2015-02-10 22:48:58,928 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:58,929 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:58,961 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:58,962 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:58,962 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:58,962 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:58,963 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:58,964 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:58,964 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:58,970 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:58,971 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 22:48:58,972 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:58,973 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:58,974 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,974 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:58,975 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:58,976 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:58,977 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:58,977 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:58,977 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:58,977 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,033 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,036 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1104083515_0033_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,038 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,038 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1104083515_0033_m_000000_0' done.
2015-02-10 22:48:59,038 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:59,039 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,039 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,040 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1104083515_0033_r_000000_0
2015-02-10 22:48:59,041 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,041 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7e7d611f
2015-02-10 22:48:59,042 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552520896, maxSingleShuffleLimit=138130224, mergeThreshold=364663808, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,043 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1104083515_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,045 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local1104083515_0033_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:59,045 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1104083515_0033_m_000000_0
2015-02-10 22:48:59,045 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:59,046 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,048 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,050 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,050 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,053 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,053 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:59,053 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,053 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,054 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,054 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,055 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,055 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,060 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 22:48:59,060 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][152035209/count-lhs/]
2015-02-10 22:48:59,070 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1104083515_0033_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,071 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1104083515_0033_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,072 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1104083515_0033_r_000000_0' to file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/_temporary/0/task_local1104083515_0033_r_000000
2015-02-10 22:48:59,073 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,073 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1104083515_0033_r_000000_0' done.
2015-02-10 22:48:59,073 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1104083515_0033_r_000000_0
2015-02-10 22:48:59,073 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,086 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2015-02-10 22:48:59,086 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,087 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,125 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,147 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:48:59,159 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,159 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,164 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local725792397_0034
2015-02-10 22:48:59,183 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity725792397/.staging/job_local725792397_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,184 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity725792397/.staging/job_local725792397_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,231 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local725792397_0034/job_local725792397_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,232 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local725792397_0034/job_local725792397_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,233 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,233 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local725792397_0034
2015-02-10 22:48:59,234 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,234 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,234 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,236 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,237 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,238 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,240 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:59,240 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1aa6a14b
2015-02-10 22:48:59,242 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,242 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,274 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,274 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,275 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,275 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,275 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,277 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,277 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,282 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:59,283 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 22:48:59,283 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:48:59,285 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 22:48:59,285 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:59,286 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 22:48:59,286 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 238, total: 752, max: 752
2015-02-10 22:48:59,287 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:59,288 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,288 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,288 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 22:48:59,288 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,291 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,293 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local725792397_0034_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,295 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,296 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local725792397_0034_m_000000_0' done.
2015-02-10 22:48:59,296 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,296 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,297 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,297 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local725792397_0034_r_000000_0
2015-02-10 22:48:59,299 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,299 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6a905eb5
2015-02-10 22:48:59,300 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552658496, maxSingleShuffleLimit=138164624, mergeThreshold=364754624, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,301 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local725792397_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,302 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local725792397_0034_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 22:48:59,303 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local725792397_0034_m_000000_0
2015-02-10 22:48:59,303 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 22:48:59,304 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,305 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,307 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,309 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,309 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,310 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,311 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 22:48:59,311 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,311 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,312 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 22:48:59,313 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,313 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,313 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,319 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 22:48:59,320 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][7584469547/count-rhs/]
2015-02-10 22:48:59,329 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local725792397_0034_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:48:59,331 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local725792397_0034_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,332 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local725792397_0034_r_000000_0' to file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/_temporary/0/task_local725792397_0034_r_000000
2015-02-10 22:48:59,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,333 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local725792397_0034_r_000000_0' done.
2015-02-10 22:48:59,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local725792397_0034_r_000000_0
2015-02-10 22:48:59,333 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,349 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2015-02-10 22:48:59,350 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,351 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,393 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,406 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,427 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:59,437 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,437 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,443 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1447370509_0035
2015-02-10 22:48:59,462 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1447370509/.staging/job_local1447370509_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,463 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1447370509/.staging/job_local1447370509_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,517 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1447370509_0035/job_local1447370509_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,518 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1447370509_0035/job_local1447370509_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,519 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,520 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1447370509_0035
2015-02-10 22:48:59,520 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,520 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,521 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,523 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,523 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,525 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,526 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000
2015-02-10 22:48:59,527 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@289dc0a8
2015-02-10 22:48:59,528 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,528 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,558 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,558 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,558 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,558 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,558 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,561 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,567 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5847483701/source/]
2015-02-10 22:48:59,567 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,569 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:59,570 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,570 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,570 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,570 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,573 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,574 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,576 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5847483701_source_77D070BA7154495FB0F0EE34ABC63BF8/part-00000:0+160
2015-02-10 22:48:59,576 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_m_000000_0' done.
2015-02-10 22:48:59,576 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,576 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,578 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,579 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/part-00000
2015-02-10 22:48:59,579 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@612bd56c
2015-02-10 22:48:59,580 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,580 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,604 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,604 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,604 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,604 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,604 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,606 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,606 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,611 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][7584469547/count-rhs/]
2015-02-10 22:48:59,611 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,614 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:59,614 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,614 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,614 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,614 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,617 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,619 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:59,620 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/7584469547_count_rhs_9A39F6A87410482384A9CB4D394AF522/part-00000:0+160
2015-02-10 22:48:59,620 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_m_000001_0' done.
2015-02-10 22:48:59,621 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,621 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,621 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,622 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1447370509_0035_r_000000_0
2015-02-10 22:48:59,623 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,623 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@674009d2
2015-02-10 22:48:59,624 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552153920, maxSingleShuffleLimit=138038480, mergeThreshold=364421600, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,624 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1447370509_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,627 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1447370509_0035_m_000000_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,627 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1447370509_0035_m_000000_0
2015-02-10 22:48:59,628 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 22:48:59,630 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1447370509_0035_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,630 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1447370509_0035_m_000001_0
2015-02-10 22:48:59,630 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2015-02-10 22:48:59,631 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,632 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,632 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,634 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:59,634 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2015-02-10 22:48:59,635 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,636 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 156 bytes from disk
2015-02-10 22:48:59,636 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,636 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,637 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2015-02-10 22:48:59,637 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,638 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,638 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,643 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 22:48:59,643 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][4454370365/source_count-rhs/]
2015-02-10 22:48:59,658 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,658 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,665 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1447370509_0035_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,666 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,667 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1447370509_0035_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,667 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1447370509_0035_r_000000_0' to file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/_temporary/0/task_local1447370509_0035_r_000000
2015-02-10 22:48:59,668 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,668 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1447370509_0035_r_000000_0' done.
2015-02-10 22:48:59,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1447370509_0035_r_000000_0
2015-02-10 22:48:59,669 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:48:59,685 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2015-02-10 22:48:59,686 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,687 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:48:59,729 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,743 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:48:59,763 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:48:59,773 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:48:59,773 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:48:59,779 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1541484361_0036
2015-02-10 22:48:59,806 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1541484361/.staging/job_local1541484361_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,807 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1541484361/.staging/job_local1541484361_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,855 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1541484361_0036/job_local1541484361_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:48:59,857 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1541484361_0036/job_local1541484361_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:48:59,857 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:48:59,857 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1541484361_0036
2015-02-10 22:48:59,858 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 22:48:59,858 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:48:59,858 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:48:59,862 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:48:59,862 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,863 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,864 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/part-00000
2015-02-10 22:48:59,864 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@11e71d39
2015-02-10 22:48:59,866 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,866 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,890 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,890 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,890 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,890 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,890 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,892 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,893 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,900 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][4454370365/source_count-rhs/]
2015-02-10 22:48:59,900 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,902 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:59,902 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,903 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,903 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 22:48:59,903 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,905 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,907 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_m_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,909 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4454370365_source_count_rhs_FBF7F87AC9FE454E807704D45E0EB6C7/part-00000:0+200
2015-02-10 22:48:59,909 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_m_000000_0' done.
2015-02-10 22:48:59,909 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,909 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,910 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,912 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/part-00000
2015-02-10 22:48:59,912 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7ccf9a33
2015-02-10 22:48:59,913 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:48:59,914 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:48:59,937 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:48:59,937 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:48:59,938 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:48:59,938 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:48:59,938 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:48:59,939 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:48:59,939 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][152035209/count-lhs/]
2015-02-10 22:48:59,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:48:59,950 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:48:59,950 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:48:59,950 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 22:48:59,950 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:48:59,953 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:48:59,955 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_m_000001_0 is done. And is in the process of committing
2015-02-10 22:48:59,956 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/152035209_count_lhs_856A3A2E025F419F972FB242029AC240/part-00000:0+160
2015-02-10 22:48:59,956 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_m_000001_0' done.
2015-02-10 22:48:59,956 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,957 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:48:59,957 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:48:59,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1541484361_0036_r_000000_0
2015-02-10 22:48:59,959 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:48:59,959 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5ee8730f
2015-02-10 22:48:59,960 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552199744, maxSingleShuffleLimit=138049936, mergeThreshold=364451840, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:48:59,961 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1541484361_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:48:59,963 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1541484361_0036_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 22:48:59,964 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1541484361_0036_m_000000_0
2015-02-10 22:48:59,964 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 22:48:59,967 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1541484361_0036_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 22:48:59,968 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1541484361_0036_m_000001_0
2015-02-10 22:48:59,968 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 117, usedMemory ->194
2015-02-10 22:48:59,969 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:48:59,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,970 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:48:59,972 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:48:59,973 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2015-02-10 22:48:59,974 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2015-02-10 22:48:59,975 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2015-02-10 22:48:59,975 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:48:59,975 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:48:59,976 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2015-02-10 22:48:59,977 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,977 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:48:59,977 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:48:59,984 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 22:48:59,984 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 22:48:59,992 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,992 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 22:48:59,995 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1541484361_0036_r_000000_0 is done. And is in the process of committing
2015-02-10 22:48:59,996 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:48:59,996 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1541484361_0036_r_000000_0 is allowed to commit now
2015-02-10 22:48:59,997 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1541484361_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local1541484361_0036_r_000000
2015-02-10 22:48:59,998 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:48:59,998 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1541484361_0036_r_000000_0' done.
2015-02-10 22:48:59,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1541484361_0036_r_000000_0
2015-02-10 22:48:59,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,019 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2015-02-10 22:49:00,052 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,086 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,089 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2015-02-10 22:49:00,123 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,124 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,132 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,135 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,135 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,143 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting
2015-02-10 22:49:00,144 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:49:00,144 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,144 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 22:49:00,144 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] parallel execution is enabled: false
2015-02-10 22:49:00,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting jobs: 1
2015-02-10 22:49:00,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] allocating threads: 1
2015-02-10 22:49:00,146 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2015-02-10 22:49:00,147 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,148 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,173 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,174 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,192 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 22:49:00,208 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1200652964_0037
2015-02-10 22:49:00,226 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1200652964/.staging/job_local1200652964_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,228 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1200652964/.staging/job_local1200652964_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,273 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1200652964_0037/job_local1200652964_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,274 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1200652964_0037/job_local1200652964_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,275 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,275 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local1200652964_0037
2015-02-10 22:49:00,275 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,275 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2015-02-10 22:49:00,276 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,279 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,280 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,281 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,281 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@43798b1a
2015-02-10 22:49:00,282 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,283 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,304 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,304 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,304 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,304 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,304 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,306 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,310 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,311 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,317 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:49:00,317 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,317 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,317 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,317 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,320 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,321 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:00,323 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_m_000000_0' done.
2015-02-10 22:49:00,323 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,324 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,325 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,326 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt
2015-02-10 22:49:00,326 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@df61ca
2015-02-10 22:49:00,327 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,327 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,358 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,358 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,358 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,358 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,358 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,360 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,360 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt"]
2015-02-10 22:49:00,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,368 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:49:00,368 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,368 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,368 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,368 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,371 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,372 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_m_000001_0 is done. And is in the process of committing
2015-02-10 22:49:00,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 22:49:00,374 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_m_000001_0' done.
2015-02-10 22:49:00,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:00,375 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:00,375 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1200652964_0037_r_000000_0
2015-02-10 22:49:00,376 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,377 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6e7c6928
2015-02-10 22:49:00,377 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552429120, maxSingleShuffleLimit=138107280, mergeThreshold=364603232, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:00,379 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1200652964_0037_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:00,383 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local1200652964_0037_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,384 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1200652964_0037_m_000000_0
2015-02-10 22:49:00,384 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:49:00,386 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local1200652964_0037_m_000001_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,386 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1200652964_0037_m_000001_0
2015-02-10 22:49:00,386 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2015-02-10 22:49:00,387 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:00,388 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,389 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:00,391 INFO mapred.Merger (Merger.java:merge(591)) - Merging 2 sorted segments
2015-02-10 22:49:00,391 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2015-02-10 22:49:00,392 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:00,393 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 176 bytes from disk
2015-02-10 22:49:00,393 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:00,393 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,393 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2015-02-10 22:49:00,394 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,394 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:00,394 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,399 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 22:49:00,399 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 22:49:00,413 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1200652964_0037_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,414 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 22:49:00,414 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1200652964_0037_r_000000_0 is allowed to commit now
2015-02-10 22:49:00,415 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1200652964_0037_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local1200652964_0037_r_000000
2015-02-10 22:49:00,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:00,416 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1200652964_0037_r_000000_0' done.
2015-02-10 22:49:00,416 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1200652964_0037_r_000000_0
2015-02-10 22:49:00,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,439 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2015-02-10 22:49:00,468 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,470 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2015-02-10 22:49:00,490 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,490 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,496 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,498 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,499 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,503 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 22:49:00,503 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,504 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 22:49:00,504 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] parallel execution is enabled: false
2015-02-10 22:49:00,505 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting jobs: 1
2015-02-10 22:49:00,505 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] allocating threads: 1
2015-02-10 22:49:00,506 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2015-02-10 22:49:00,507 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,508 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,533 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,554 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:49:00,570 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1535422660_0038
2015-02-10 22:49:00,586 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1535422660/.staging/job_local1535422660_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,587 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity1535422660/.staging/job_local1535422660_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,637 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1535422660_0038/job_local1535422660_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,638 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1535422660_0038/job_local1535422660_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,638 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,639 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1535422660_0038
2015-02-10 22:49:00,639 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 22:49:00,639 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,640 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,644 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,645 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,646 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5295c398
2015-02-10 22:49:00,647 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,647 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,682 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,682 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,682 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,682 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,682 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,684 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,684 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,688 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,688 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 22:49:00,692 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:49:00,692 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:00,692 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:00,692 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 22:49:00,692 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 22:49:00,695 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:00,697 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1535422660_0038_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,699 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:00,699 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1535422660_0038_m_000000_0' done.
2015-02-10 22:49:00,699 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,699 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:00,700 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:00,700 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1535422660_0038_r_000000_0
2015-02-10 22:49:00,701 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,701 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@70afe432
2015-02-10 22:49:00,702 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=550548288, maxSingleShuffleLimit=137637072, mergeThreshold=363361888, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:00,703 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1535422660_0038_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:00,706 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#31 about to shuffle output of map attempt_local1535422660_0038_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 22:49:00,707 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1535422660_0038_m_000000_0
2015-02-10 22:49:00,707 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 22:49:00,707 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:00,708 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,709 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:00,710 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,710 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:49:00,712 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:00,712 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 22:49:00,712 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:00,712 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:00,713 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 22:49:00,713 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,713 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:00,714 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,717 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 22:49:00,717 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 22:49:00,727 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local1535422660_0038_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:00,728 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:00,728 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local1535422660_0038_r_000000_0 is allowed to commit now
2015-02-10 22:49:00,729 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1535422660_0038_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local1535422660_0038_r_000000
2015-02-10 22:49:00,730 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:00,730 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1535422660_0038_r_000000_0' done.
2015-02-10 22:49:00,730 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1535422660_0038_r_000000_0
2015-02-10 22:49:00,731 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:00,746 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2015-02-10 22:49:00,779 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,781 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2015-02-10 22:49:00,806 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 22:49:00,806 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 22:49:00,813 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 22:49:00,815 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 22:49:00,816 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1009)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 22:49:00,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 22:49:00,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:00,821 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 22:49:00,822 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] parallel execution is enabled: false
2015-02-10 22:49:00,822 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting jobs: 1
2015-02-10 22:49:00,822 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [count] allocating threads: 1
2015-02-10 22:49:00,823 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2015-02-10 22:49:00,824 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,825 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 22:49:00,848 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:00,868 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 22:49:00,884 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local929874522_0039
2015-02-10 22:49:00,901 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity929874522/.staging/job_local929874522_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,902 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/tmp/cascading/staging/teamcity929874522/.staging/job_local929874522_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,949 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local929874522_0039/job_local929874522_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-10 22:49:00,951 WARN conf.Configuration (Configuration.java:loadProperty(2358)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local929874522_0039/job_local929874522_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-10 22:49:00,951 INFO mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 22:49:00,952 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local929874522_0039
2015-02-10 22:49:00,952 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 22:49:00,952 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 22:49:00,952 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 22:49:00,955 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 22:49:00,955 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:00,957 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:00,959 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt
2015-02-10 22:49:00,959 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@37ba658a
2015-02-10 22:49:00,960 INFO mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 22:49:00,961 INFO mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 22:49:00,990 INFO mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 22:49:00,991 INFO mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 22:49:00,991 INFO mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 22:49:00,991 INFO mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 22:49:00,991 INFO mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 22:49:00,992 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 22:49:00,992 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 22:49:00,999 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt"]
2015-02-10 22:49:01,000 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:49:01,000 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 22:49:01,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-10 22:49:01,008 INFO mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-10 22:49:01,008 INFO mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-10 22:49:01,008 INFO mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2015-02-10 22:49:01,008 INFO mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 22:49:01,010 INFO mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-10 22:49:01,012 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local929874522_0039_m_000000_0 is done. And is in the process of committing
2015-02-10 22:49:01,014 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 22:49:01,014 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local929874522_0039_m_000000_0' done.
2015-02-10 22:49:01,014 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:01,014 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 22:49:01,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 22:49:01,015 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local929874522_0039_r_000000_0
2015-02-10 22:49:01,017 INFO mapred.Task (Task.java:initialize(581)) - Using ResourceCalculatorProcessTree : [ ]
2015-02-10 22:49:01,017 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@33319df3
2015-02-10 22:49:01,017 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552337408, maxSingleShuffleLimit=138084352, mergeThreshold=364542688, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 22:49:01,018 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local929874522_0039_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 22:49:01,019 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#32 about to shuffle output of map attempt_local929874522_0039_m_000000_0 decomp: 10 len: 14 to MEMORY
2015-02-10 22:49:01,020 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local929874522_0039_m_000000_0
2015-02-10 22:49:01,020 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2015-02-10 22:49:01,021 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 22:49:01,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,022 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 22:49:01,024 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:01,025 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 22:49:01,026 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2015-02-10 22:49:01,026 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 14 bytes from disk
2015-02-10 22:49:01,026 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 22:49:01,026 INFO mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-10 22:49:01,027 INFO mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 22:49:01,027 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,028 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 22:49:01,028 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 22:49:01,034 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 22:49:01,034 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 22:49:01,044 INFO mapred.Task (Task.java:done(995)) - Task:attempt_local929874522_0039_r_000000_0 is done. And is in the process of committing
2015-02-10 22:49:01,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 22:49:01,045 INFO mapred.Task (Task.java:commit(1156)) - Task attempt_local929874522_0039_r_000000_0 is allowed to commit now
2015-02-10 22:49:01,046 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local929874522_0039_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local929874522_0039_r_000000
2015-02-10 22:49:01,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 22:49:01,047 INFO mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local929874522_0039_r_000000_0' done.
2015-02-10 22:49:01,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local929874522_0039_r_000000_0
2015-02-10 22:49:01,047 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 22:49:01,066 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2015-02-10 22:49:01,096 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:01,125 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-10 22:49:01,127 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.4.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull