Class cascading.pipe.assembly.AssemblyHelpersPlatformTest

33

tests

0

failures

19.397s

duration

100%

successful

Tests

Test Duration Result
testAverageBy 0.341s passed
testAverageByNull 0.355s passed
testAverageMerge 0.934s passed
testCoerce 0.230s passed
testCoerceFields 0.285s passed
testCount 0.504s passed
testCountAll 0.319s passed
testCountCount 0.961s passed
testCountMerge 0.762s passed
testCountNullNotNull 0.327s passed
testDiscardNarrow 0.251s passed
testFirstBy 0.510s passed
testFirstByWithoutComparator 0.310s passed
testMaxBy 0.422s passed
testMaxByNullSafety 0.339s passed
testMaxByString 0.537s passed
testMinBy 0.412s passed
testMinByNullSafety 0.756s passed
testMinByString 0.689s passed
testParallelAggregates 0.358s passed
testParallelAggregatesMerge 3.517s passed
testParallelAggregatesPriorMerge 0.442s passed
testRenameAll 0.684s passed
testRenameNamed 0.260s passed
testRenameNarrow 0.259s passed
testRetainNarrow 0.234s passed
testSameSourceMerge 0.758s passed
testSameSourceMergeThreeWay 1.840s passed
testSumBy 0.361s passed
testSumByNulls 0.351s passed
testSumMerge 0.463s passed
testUnique 0.274s passed
testUniqueMerge 0.352s passed

Standard output

2015-02-10 20:56:57,455 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-10 20:56:57,522 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-10 20:56:57,522 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2015-02-10 20:56:57,573 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-10 20:56:58,247 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-10 20:56:58,831 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:56:58,834 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:56:58,846 INFO  property.AppProps (AppProps.java:getAppID(169)) - using app.id: 0B1F1D200D41425E936299F908462FFC
2015-02-10 20:56:58,944 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:56:58,985 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:56:58,988 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:56:59,138 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-10 20:56:59,141 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:56:59,142 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:56:59,143 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:56:59,143 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 20:56:59,145 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 20:56:59,146 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 20:56:59,146 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 20:56:59,148 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2015-02-10 20:56:59,172 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-10 20:56:59,173 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-10 20:56:59,198 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:56:59,502 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:56:59,513 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:56:59,544 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:56:59,708 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1611015894_0001
2015-02-10 20:56:59,753 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1611015894/.staging/job_local1611015894_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:56:59,755 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1611015894/.staging/job_local1611015894_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:56:59,942 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1611015894_0001/job_local1611015894_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:56:59,943 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1611015894_0001/job_local1611015894_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:56:59,955 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:56:59,959 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:56:59,959 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1611015894_0001
2015-02-10 20:56:59,959 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:56:59,962 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:00,032 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:00,035 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,076 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,086 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:00,086 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@55291cd
2015-02-10 20:57:00,095 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:00,102 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:00,254 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:00,255 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:00,255 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:00,255 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:00,255 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:00,280 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:00,281 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,342 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-10 20:57:00,349 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:00,349 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,353 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:00,368 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:00,368 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,368 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,370 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:00,370 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,370 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,371 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:00,372 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,372 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:00,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 202, max: 672
2015-02-10 20:57:00,373 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,378 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:00,378 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:00,379 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:00,379 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:00,379 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:00,391 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:00,395 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:00,406 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:00,406 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_m_000000_0' done.
2015-02-10 20:57:00,407 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,407 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,409 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,411 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:00,412 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@47098a
2015-02-10 20:57:00,414 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:00,414 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:00,563 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:00,564 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:00,564 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:00,564 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:00,564 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:00,568 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:00,568 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,594 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:00,594 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,595 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:00,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:00,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,693 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:00,693 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,693 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,694 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:00,695 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,695 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,696 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:00,696 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 85, total: 302, max: 672
2015-02-10 20:57:00,696 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 44% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:00,698 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:00,698 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:00,698 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:00,698 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:00,699 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:00,728 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:00,732 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:00,734 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:00,735 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_m_000001_0' done.
2015-02-10 20:57:00,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:00,738 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:00,738 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1611015894_0001_r_000000_0
2015-02-10 20:57:00,747 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:00,750 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3739491b
2015-02-10 20:57:00,764 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:00,768 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1611015894_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:00,799 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local1611015894_0001_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:00,804 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1611015894_0001_m_000000_0
2015-02-10 20:57:00,845 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 20:57:00,849 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local1611015894_0001_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:00,850 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1611015894_0001_m_000001_0
2015-02-10 20:57:00,850 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 20:57:00,851 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:00,852 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,853 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:00,863 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:00,863 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 20:57:00,867 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:00,867 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 20:57:00,868 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:00,868 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:00,869 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 20:57:00,870 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,872 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:00,873 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:00,900 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2015-02-10 20:57:00,900 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false"]
2015-02-10 20:57:00,929 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1611015894_0001_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:00,931 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:00,931 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1611015894_0001_r_000000_0 is allowed to commit now
2015-02-10 20:57:00,933 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1611015894_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary/0/task_local1611015894_0001_r_000000
2015-02-10 20:57:00,934 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:00,934 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1611015894_0001_r_000000_0' done.
2015-02-10 20:57:00,935 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1611015894_0001_r_000000_0
2015-02-10 20:57:00,935 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:00,987 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+false/_temporary
2015-02-10 20:57:01,042 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,083 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,087 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+false
2015-02-10 20:57:01,124 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:01,124 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:01,141 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:01,147 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:01,148 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:01,164 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:01,165 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,166 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 20:57:01,167 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 20:57:01,167 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 20:57:01,168 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 20:57:01,169 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2015-02-10 20:57:01,171 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,173 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,208 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,234 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:01,271 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local532599968_0002
2015-02-10 20:57:01,293 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity532599968/.staging/job_local532599968_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:01,294 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity532599968/.staging/job_local532599968_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:01,386 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local532599968_0002/job_local532599968_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:01,386 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local532599968_0002/job_local532599968_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:01,388 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:01,388 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local532599968_0002
2015-02-10 20:57:01,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:01,389 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:01,389 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:01,394 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:01,394 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,397 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:01,400 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:01,400 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4c6030f9
2015-02-10 20:57:01,405 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:01,405 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:01,557 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:01,558 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:01,558 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:01,558 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:01,558 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:01,561 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:01,561 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:01,575 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,575 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:01,576 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:01,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:01,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 402, max: 672
2015-02-10 20:57:01,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:01,579 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:01,579 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 89, total: 402, max: 672
2015-02-10 20:57:01,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 59% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:01,582 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:01,582 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:01,582 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:01,582 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:01,582 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:01,587 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:01,590 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local532599968_0002_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:01,593 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:01,593 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local532599968_0002_m_000000_0' done.
2015-02-10 20:57:01,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,594 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:01,595 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:01,595 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local532599968_0002_r_000000_0
2015-02-10 20:57:01,599 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:01,599 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5637dde9
2015-02-10 20:57:01,600 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:01,602 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local532599968_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:01,605 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#2 about to shuffle output of map attempt_local532599968_0002_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:01,606 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local532599968_0002_m_000000_0
2015-02-10 20:57:01,606 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:01,607 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:01,609 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,609 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:01,613 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:01,614 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:01,616 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:01,617 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:01,618 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:01,618 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:01,619 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:01,620 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:01,622 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:01,635 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:01,635 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring"]
2015-02-10 20:57:01,652 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local532599968_0002_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:01,654 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:01,654 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local532599968_0002_r_000000_0 is allowed to commit now
2015-02-10 20:57:01,656 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local532599968_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary/0/task_local532599968_0002_r_000000
2015-02-10 20:57:01,658 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:01,658 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local532599968_0002_r_000000_0' done.
2015-02-10 20:57:01,659 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local532599968_0002_r_000000_0
2015-02-10 20:57:01,660 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:01,688 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbystring/_temporary
2015-02-10 20:57:01,730 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,775 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,778 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbystring
2015-02-10 20:57:01,818 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:01,819 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:01,853 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:01,862 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:01,863 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:01,879 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs] starting
2015-02-10 20:57:01,879 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:01,879 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:01,880 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 20:57:01,880 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  parallel execution is enabled: false
2015-02-10 20:57:01,880 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  starting jobs: 1
2015-02-10 20:57:01,880 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average-lhs+average-rhs]  allocating threads: 1
2015-02-10 20:57:01,881 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2015-02-10 20:57:01,882 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,884 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:01,918 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,920 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:01,944 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:01,984 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local115791930_0003
2015-02-10 20:57:02,012 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity115791930/.staging/job_local115791930_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:02,012 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity115791930/.staging/job_local115791930_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:02,086 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local115791930_0003/job_local115791930_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:02,086 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local115791930_0003/job_local115791930_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:02,088 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:02,088 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local115791930_0003
2015-02-10 20:57:02,088 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:02,090 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:02,090 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:02,096 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:02,096 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,100 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,103 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:02,103 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@12f53870
2015-02-10 20:57:02,106 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:02,107 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:02,263 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:02,264 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:02,264 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:02,265 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:02,265 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:02,269 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,270 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,286 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:02,287 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,288 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:02,290 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:02,291 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,291 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,293 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:02,293 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,294 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,295 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:02,295 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,296 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,297 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:02,297 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 73, total: 502, max: 672
2015-02-10 20:57:02,298 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 74% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:02,300 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:02,301 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:02,301 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:02,302 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:02,302 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:02,308 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:02,325 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:02,329 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:02,330 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_m_000000_0' done.
2015-02-10 20:57:02,330 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,333 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,336 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:02,336 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6f539d49
2015-02-10 20:57:02,338 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:02,339 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:02,483 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:02,484 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:02,485 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:02,485 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:02,486 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:02,487 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,488 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,504 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:02,505 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,506 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:02,511 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:02,511 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,513 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:02,513 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,515 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:02,515 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,516 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:02,517 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 54, total: 560, max: 672
2015-02-10 20:57:02,519 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:02,519 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:02,520 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:02,520 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:02,521 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:02,526 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:02,528 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:02,531 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:02,532 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_m_000001_0' done.
2015-02-10 20:57:02,533 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,533 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:02,534 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:02,534 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local115791930_0003_r_000000_0
2015-02-10 20:57:02,537 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,538 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3a5e7663
2015-02-10 20:57:02,539 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:02,541 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local115791930_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:02,543 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local115791930_0003_m_000001_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:02,544 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local115791930_0003_m_000001_0
2015-02-10 20:57:02,544 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:02,547 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#3 about to shuffle output of map attempt_local115791930_0003_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:02,548 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local115791930_0003_m_000000_0
2015-02-10 20:57:02,548 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 2, commitMemory -> 200, usedMemory ->400
2015-02-10 20:57:02,549 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:02,550 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,551 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:02,554 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:02,555 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 382 bytes
2015-02-10 20:57:02,558 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 400 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:02,558 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 402 bytes from disk
2015-02-10 20:57:02,559 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:02,560 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:02,560 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 389 bytes
2015-02-10 20:57:02,561 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,562 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:02,563 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:02,577 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2015-02-10 20:57:02,578 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage"]
2015-02-10 20:57:02,593 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local115791930_0003_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:02,595 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:02,595 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local115791930_0003_r_000000_0 is allowed to commit now
2015-02-10 20:57:02,597 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local115791930_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary/0/task_local115791930_0003_r_000000
2015-02-10 20:57:02,599 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:02,599 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local115791930_0003_r_000000_0' done.
2015-02-10 20:57:02,600 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local115791930_0003_r_000000_0
2015-02-10 20:57:02,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:02,633 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergeaverage/_temporary
2015-02-10 20:57:02,671 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,711 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,713 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergeaverage
2015-02-10 20:57:02,744 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:02,745 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:02,757 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:02,761 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:02,762 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:02,776 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:02,777 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:02,777 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 20:57:02,778 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:02,778 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:02,779 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:02,780 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2015-02-10 20:57:02,781 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:02,783 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:02,819 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:02,846 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:02,870 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local742608088_0004
2015-02-10 20:57:02,890 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity742608088/.staging/job_local742608088_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:02,891 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity742608088/.staging/job_local742608088_0004/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:02,970 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local742608088_0004/job_local742608088_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:02,971 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local742608088_0004/job_local742608088_0004.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:02,973 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:02,974 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local742608088_0004
2015-02-10 20:57:02,974 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:02,975 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:02,975 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:02,980 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:02,981 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local742608088_0004_m_000000_0
2015-02-10 20:57:02,984 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:02,986 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:02,986 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@b8235a1
2015-02-10 20:57:02,988 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:02,999 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:02,999 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,007 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:03,008 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall"]
2015-02-10 20:57:03,012 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:03,012 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local742608088_0004_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,014 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:03,014 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local742608088_0004_m_000000_0 is allowed to commit now
2015-02-10 20:57:03,015 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local742608088_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary/0/task_local742608088_0004_m_000000
2015-02-10 20:57:03,017 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:03,017 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local742608088_0004_m_000000_0' done.
2015-02-10 20:57:03,017 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local742608088_0004_m_000000_0
2015-02-10 20:57:03,019 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:03,047 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renameall/_temporary
2015-02-10 20:57:03,396 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,399 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renameall
2015-02-10 20:57:03,427 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:03,427 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:03,448 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:03,452 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:03,468 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:03,478 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:03,479 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:03,480 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 20:57:03,480 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 20:57:03,480 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 2
2015-02-10 20:57:03,480 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 20:57:03,482 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2015-02-10 20:57:03,483 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,484 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,513 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,535 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:03,554 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local613160317_0005
2015-02-10 20:57:03,573 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity613160317/.staging/job_local613160317_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:03,574 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity613160317/.staging/job_local613160317_0005/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:03,629 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local613160317_0005/job_local613160317_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:03,629 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local613160317_0005/job_local613160317_0005.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:03,630 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:03,631 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local613160317_0005
2015-02-10 20:57:03,631 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:03,631 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:03,631 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:03,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:03,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,637 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:03,639 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:03,639 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1259b924
2015-02-10 20:57:03,641 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:03,641 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:03,712 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:03,712 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:03,712 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:03,712 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:03,713 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:03,715 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:03,716 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,728 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:03,728 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:03,728 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:03,730 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:03,730 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,730 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,731 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:03,732 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,732 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,732 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:03,732 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,733 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,733 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:03,733 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 81, total: 220, max: 672
2015-02-10 20:57:03,733 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:03,735 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:03,735 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:03,736 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:03,736 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:03,740 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:03,742 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local613160317_0005_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,745 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:03,745 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local613160317_0005_m_000000_0' done.
2015-02-10 20:57:03,746 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,747 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:03,748 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:03,749 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local613160317_0005_r_000000_0
2015-02-10 20:57:03,754 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:03,754 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7ab05cd7
2015-02-10 20:57:03,755 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:03,755 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local613160317_0005_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:03,757 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#4 about to shuffle output of map attempt_local613160317_0005_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:03,758 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local613160317_0005_m_000000_0
2015-02-10 20:57:03,758 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:03,759 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:03,760 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,760 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:03,763 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:03,764 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:03,766 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:03,766 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:03,766 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:03,766 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:03,767 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:03,768 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:03,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:03,779 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:03,779 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][4331726180/count/]
2015-02-10 20:57:03,802 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:03,805 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:03,806 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 77, total: 220, max: 672
2015-02-10 20:57:03,806 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:03,806 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local613160317_0005_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:03,809 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:03,809 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local613160317_0005_r_000000_0 is allowed to commit now
2015-02-10 20:57:03,810 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local613160317_0005_r_000000_0' to file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/_temporary/0/task_local613160317_0005_r_000000
2015-02-10 20:57:03,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:03,812 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local613160317_0005_r_000000_0' done.
2015-02-10 20:57:03,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local613160317_0005_r_000000_0
2015-02-10 20:57:03,812 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:03,833 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2015-02-10 20:57:03,833 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,835 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:03,875 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:03,895 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:03,907 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:03,907 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:03,919 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1429604986_0006
2015-02-10 20:57:03,940 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1429604986/.staging/job_local1429604986_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:03,940 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1429604986/.staging/job_local1429604986_0006/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:03,996 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1429604986_0006/job_local1429604986_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:03,996 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1429604986_0006/job_local1429604986_0006.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:03,997 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:03,998 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1429604986_0006
2015-02-10 20:57:03,998 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:03,998 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:03,999 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:04,003 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:04,004 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,006 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,008 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/part-00000
2015-02-10 20:57:04,008 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@60f00e0f
2015-02-10 20:57:04,019 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,020 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,186 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,187 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,187 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,187 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,187 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,188 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,189 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,200 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][4331726180/count/]
2015-02-10 20:57:04,200 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2015-02-10 20:57:04,203 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:04,203 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,204 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,204 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 21; bufvoid = 104857600
2015-02-10 20:57:04,204 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214388(104857552); length = 9/6553600
2015-02-10 20:57:04,208 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,210 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1429604986_0006_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,212 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4331726180_count_DFACB92F77D44369982ED63DE0178E64/part-00000:0+112
2015-02-10 20:57:04,213 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1429604986_0006_m_000000_0' done.
2015-02-10 20:57:04,213 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,213 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:04,214 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:04,214 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1429604986_0006_r_000000_0
2015-02-10 20:57:04,216 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,216 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6e39b27b
2015-02-10 20:57:04,218 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:04,219 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1429604986_0006_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:04,221 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#5 about to shuffle output of map attempt_local1429604986_0006_m_000000_0 decomp: 29 len: 33 to MEMORY
2015-02-10 20:57:04,222 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 29 bytes from map-output for attempt_local1429604986_0006_m_000000_0
2015-02-10 20:57:04,222 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->29
2015-02-10 20:57:04,222 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:04,223 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,224 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:04,230 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,230 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 20:57:04,232 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 29 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:04,233 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 33 bytes from disk
2015-02-10 20:57:04,233 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:04,233 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,234 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 24 bytes
2015-02-10 20:57:04,235 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,236 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:04,236 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,248 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2015-02-10 20:57:04,248 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount"]
2015-02-10 20:57:04,259 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1429604986_0006_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,261 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:04,261 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1429604986_0006_r_000000_0 is allowed to commit now
2015-02-10 20:57:04,262 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1429604986_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary/0/task_local1429604986_0006_r_000000
2015-02-10 20:57:04,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:04,266 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1429604986_0006_r_000000_0' done.
2015-02-10 20:57:04,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1429604986_0006_r_000000_0
2015-02-10 20:57:04,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:04,292 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countcount/_temporary
2015-02-10 20:57:04,326 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,359 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,361 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countcount
2015-02-10 20:57:04,391 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:04,391 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:04,408 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:04,412 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:04,413 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:04,425 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs] starting
2015-02-10 20:57:04,426 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:04,426 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:04,426 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 20:57:04,427 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  parallel execution is enabled: false
2015-02-10 20:57:04,427 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  starting jobs: 1
2015-02-10 20:57:04,427 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs+count-rhs]  allocating threads: 1
2015-02-10 20:57:04,427 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2015-02-10 20:57:04,428 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:04,430 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:04,458 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,460 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:04,480 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:04,499 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local606478226_0007
2015-02-10 20:57:04,522 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity606478226/.staging/job_local606478226_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:04,523 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity606478226/.staging/job_local606478226_0007/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:04,583 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local606478226_0007/job_local606478226_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:04,583 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local606478226_0007/job_local606478226_0007.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:04,584 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:04,584 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local606478226_0007
2015-02-10 20:57:04,585 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:04,585 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:04,585 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:04,588 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:04,588 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,590 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,592 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:04,592 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6d9dd520
2015-02-10 20:57:04,594 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,594 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,740 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,740 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,740 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,741 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,741 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,743 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,743 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,757 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:04,757 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:04,758 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:04,760 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:04,760 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,760 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,761 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:04,761 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,761 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,762 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:04,762 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,762 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,763 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:04,763 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 114, total: 431, max: 672
2015-02-10 20:57:04,763 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 64% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,765 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:04,765 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,765 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,765 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:04,766 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:04,770 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,772 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:04,775 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:04,775 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_m_000000_0' done.
2015-02-10 20:57:04,775 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,775 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,780 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,782 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:04,782 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@642b6fc7
2015-02-10 20:57:04,784 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:04,784 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:04,928 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:04,928 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:04,929 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:04,929 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:04,929 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:04,931 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:04,931 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:04,946 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:04,946 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:04,947 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:04,951 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:04,951 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,952 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,953 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:04,953 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,953 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,954 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:04,954 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,954 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,955 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:04,955 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 113, total: 531, max: 672
2015-02-10 20:57:04,955 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:04,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:04,957 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:04,957 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:04,957 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:04,957 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:04,961 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:04,964 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:04,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:04,966 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_m_000001_0' done.
2015-02-10 20:57:04,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,967 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:04,967 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:04,967 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local606478226_0007_r_000000_0
2015-02-10 20:57:04,970 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:04,971 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4aa4ceeb
2015-02-10 20:57:04,972 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:04,973 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local606478226_0007_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:04,976 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local606478226_0007_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:04,977 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local606478226_0007_m_000000_0
2015-02-10 20:57:04,977 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:04,980 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#6 about to shuffle output of map attempt_local606478226_0007_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:04,981 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local606478226_0007_m_000001_0
2015-02-10 20:57:04,981 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 20:57:04,981 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:04,982 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:04,983 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:04,985 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:04,985 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 20:57:04,988 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:04,988 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 20:57:04,988 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:04,988 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:04,989 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 20:57:04,990 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:04,990 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:04,991 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,004 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:05,004 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount"]
2015-02-10 20:57:05,017 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local606478226_0007_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,019 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:05,019 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local606478226_0007_r_000000_0 is allowed to commit now
2015-02-10 20:57:05,020 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local606478226_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary/0/task_local606478226_0007_r_000000
2015-02-10 20:57:05,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:05,022 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local606478226_0007_r_000000_0' done.
2015-02-10 20:57:05,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local606478226_0007_r_000000_0
2015-02-10 20:57:05,022 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:05,053 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergecount/_temporary
2015-02-10 20:57:05,086 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,122 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,124 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergecount
2015-02-10 20:57:05,149 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:05,150 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:05,161 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:05,164 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:05,165 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:05,175 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 20:57:05,175 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,175 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 20:57:05,176 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-10 20:57:05,176 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-10 20:57:05,176 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-10 20:57:05,177 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...splatform/minbynullsafety
2015-02-10 20:57:05,177 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,178 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,207 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,228 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:05,246 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local882655297_0008
2015-02-10 20:57:05,265 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity882655297/.staging/job_local882655297_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:05,266 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity882655297/.staging/job_local882655297_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:05,331 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882655297_0008/job_local882655297_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:05,331 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local882655297_0008/job_local882655297_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:05,333 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:05,334 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:05,334 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:05,334 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local882655297_0008
2015-02-10 20:57:05,335 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 20:57:05,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:05,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,341 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:05,343 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:05,343 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@620a9239
2015-02-10 20:57:05,344 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:05,345 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:05,671 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:05,671 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:05,671 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:05,672 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:05,672 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:05,673 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:05,673 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,684 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,685 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:05,685 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:05,687 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:05,687 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,687 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,688 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:05,689 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,689 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:05,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,690 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:05,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 148, total: 254, max: 672
2015-02-10 20:57:05,691 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 37% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:05,693 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:05,694 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:05,694 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:05,694 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:05,694 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:05,737 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:05,739 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local882655297_0008_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,742 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:05,742 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882655297_0008_m_000000_0' done.
2015-02-10 20:57:05,742 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,742 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:05,743 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:05,743 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local882655297_0008_r_000000_0
2015-02-10 20:57:05,745 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:05,745 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@bab5e85
2015-02-10 20:57:05,746 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:05,747 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local882655297_0008_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:05,749 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#7 about to shuffle output of map attempt_local882655297_0008_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:05,749 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local882655297_0008_m_000000_0
2015-02-10 20:57:05,749 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:05,750 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:05,751 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,752 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:05,754 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:05,754 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:05,756 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:05,756 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:05,756 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:05,757 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:05,757 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:05,758 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,758 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:05,758 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:05,768 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:05,769 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety"]
2015-02-10 20:57:05,780 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local882655297_0008_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:05,782 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:05,782 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local882655297_0008_r_000000_0 is allowed to commit now
2015-02-10 20:57:05,783 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local882655297_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary/0/task_local882655297_0008_r_000000
2015-02-10 20:57:05,785 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:05,785 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local882655297_0008_r_000000_0' done.
2015-02-10 20:57:05,785 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local882655297_0008_r_000000_0
2015-02-10 20:57:05,785 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:05,810 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minbynullsafety/_temporary
2015-02-10 20:57:05,843 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,878 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,880 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minbynullsafety
2015-02-10 20:57:05,905 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:05,905 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:05,915 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:05,917 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:05,918 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 20:57:05,929 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 20:57:05,930 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2015-02-10 20:57:05,931 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,932 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:05,959 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:05,981 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:05,998 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1776200430_0009
2015-02-10 20:57:06,016 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1776200430/.staging/job_local1776200430_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:06,016 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1776200430/.staging/job_local1776200430_0009/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:06,069 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1776200430_0009/job_local1776200430_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:06,070 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1776200430_0009/job_local1776200430_0009.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:06,072 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:06,072 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1776200430_0009
2015-02-10 20:57:06,072 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:06,072 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:06,073 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:06,076 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:06,076 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,079 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,081 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:06,081 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@49f8a4fd
2015-02-10 20:57:06,083 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:06,083 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:06,251 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:06,252 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:06,252 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:06,252 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:06,252 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:06,253 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:06,254 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,263 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:06,263 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:06,264 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:06,266 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:06,266 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 354, max: 672
2015-02-10 20:57:06,266 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,267 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:06,267 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 101, total: 354, max: 672
2015-02-10 20:57:06,267 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,269 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:06,269 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:06,269 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:06,269 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:06,269 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:06,273 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:06,275 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1776200430_0009_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,277 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:06,277 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1776200430_0009_m_000000_0' done.
2015-02-10 20:57:06,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:06,278 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:06,279 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1776200430_0009_r_000000_0
2015-02-10 20:57:06,280 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,281 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2587b08d
2015-02-10 20:57:06,281 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:06,282 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1776200430_0009_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:06,284 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#8 about to shuffle output of map attempt_local1776200430_0009_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:06,285 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1776200430_0009_m_000000_0
2015-02-10 20:57:06,286 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:06,287 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:06,288 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,288 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:06,290 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,290 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:06,292 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:06,292 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:06,292 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:06,293 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,293 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:06,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,294 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:06,294 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,303 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2015-02-10 20:57:06,304 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring"]
2015-02-10 20:57:06,315 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1776200430_0009_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,317 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,317 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1776200430_0009_r_000000_0 is allowed to commit now
2015-02-10 20:57:06,318 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1776200430_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary/0/task_local1776200430_0009_r_000000
2015-02-10 20:57:06,320 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:06,320 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1776200430_0009_r_000000_0' done.
2015-02-10 20:57:06,320 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1776200430_0009_r_000000_0
2015-02-10 20:57:06,321 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:06,345 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbystring/_temporary
2015-02-10 20:57:06,377 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,416 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,418 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbystring
2015-02-10 20:57:06,453 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:06,454 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:06,468 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:06,471 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:06,472 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:06,488 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 20:57:06,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:57:06,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 20:57:06,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-10 20:57:06,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-10 20:57:06,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-10 20:57:06,490 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2015-02-10 20:57:06,491 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,492 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,521 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,542 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:06,563 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1121178405_0010
2015-02-10 20:57:06,585 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1121178405/.staging/job_local1121178405_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:06,585 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1121178405/.staging/job_local1121178405_0010/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:06,638 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1121178405_0010/job_local1121178405_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:06,639 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1121178405_0010/job_local1121178405_0010.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:06,640 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:06,640 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local1121178405_0010
2015-02-10 20:57:06,640 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:06,640 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 20:57:06,642 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:06,645 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:06,646 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,648 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,650 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt
2015-02-10 20:57:06,650 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5054c2b8
2015-02-10 20:57:06,652 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:06,652 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:06,797 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:06,797 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:06,798 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:06,798 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:06,798 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:06,802 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:06,802 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,812 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt"]
2015-02-10 20:57:06,813 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:06,813 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:06,817 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:06,817 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 130, total: 454, max: 672
2015-02-10 20:57:06,818 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,819 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:06,820 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 130, total: 454, max: 672
2015-02-10 20:57:06,820 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 67% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:06,822 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:06,823 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:06,823 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:06,823 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 170; bufvoid = 104857600
2015-02-10 20:57:06,824 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:06,827 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:06,829 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1121178405_0010_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,832 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2015-02-10 20:57:06,832 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1121178405_0010_m_000000_0' done.
2015-02-10 20:57:06,832 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,833 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:06,833 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:06,833 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1121178405_0010_r_000000_0
2015-02-10 20:57:06,835 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:06,836 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7e628e42
2015-02-10 20:57:06,836 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:06,837 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1121178405_0010_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:06,839 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#9 about to shuffle output of map attempt_local1121178405_0010_m_000000_0 decomp: 182 len: 186 to MEMORY
2015-02-10 20:57:06,840 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 182 bytes from map-output for attempt_local1121178405_0010_m_000000_0
2015-02-10 20:57:06,840 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 182, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->182
2015-02-10 20:57:06,841 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:06,842 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,842 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:06,845 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,845 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 20:57:06,846 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 182 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:06,847 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 186 bytes from disk
2015-02-10 20:57:06,847 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:06,847 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:06,848 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 160 bytes
2015-02-10 20:57:06,848 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,849 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:06,849 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:06,858 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:06,859 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields"]
2015-02-10 20:57:06,870 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1121178405_0010_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:06,872 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:06,872 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1121178405_0010_r_000000_0 is allowed to commit now
2015-02-10 20:57:06,873 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1121178405_0010_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary/0/task_local1121178405_0010_r_000000
2015-02-10 20:57:06,875 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:06,875 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1121178405_0010_r_000000_0' done.
2015-02-10 20:57:06,875 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1121178405_0010_r_000000_0
2015-02-10 20:57:06,875 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:06,895 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfields/_temporary
2015-02-10 20:57:06,926 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:06,928 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfields
2015-02-10 20:57:06,954 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:06,955 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:06,963 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:06,965 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:06,966 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:06,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 20:57:06,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:06,976 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 20:57:06,976 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-10 20:57:06,976 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-10 20:57:06,976 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-10 20:57:06,978 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2015-02-10 20:57:06,979 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:06,980 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,006 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,026 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,042 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1599758861_0011
2015-02-10 20:57:07,060 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1599758861/.staging/job_local1599758861_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,061 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1599758861/.staging/job_local1599758861_0011/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,114 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1599758861_0011/job_local1599758861_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,114 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1599758861_0011/job_local1599758861_0011.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,115 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,116 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local1599758861_0011
2015-02-10 20:57:07,116 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,116 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 20:57:07,117 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,119 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,120 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1599758861_0011_m_000000_0
2015-02-10 20:57:07,122 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,123 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:07,124 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@38128f03
2015-02-10 20:57:07,125 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:07,133 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,133 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,143 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:07,143 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields"]
2015-02-10 20:57:07,154 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:07,154 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1599758861_0011_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,155 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:07,156 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1599758861_0011_m_000000_0 is allowed to commit now
2015-02-10 20:57:07,156 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1599758861_0011_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary/0/task_local1599758861_0011_m_000000
2015-02-10 20:57:07,158 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:07,158 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1599758861_0011_m_000000_0' done.
2015-02-10 20:57:07,158 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1599758861_0011_m_000000_0
2015-02-10 20:57:07,158 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,182 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coercefields/_temporary
2015-02-10 20:57:07,212 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,214 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coercefields
2015-02-10 20:57:07,241 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:07,242 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:07,253 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:07,255 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:07,256 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:07,263 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:07,263 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,263 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 20:57:07,264 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 20:57:07,264 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 20:57:07,264 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 20:57:07,264 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2015-02-10 20:57:07,265 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,267 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,297 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,319 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,334 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local518708628_0012
2015-02-10 20:57:07,357 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity518708628/.staging/job_local518708628_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,357 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity518708628/.staging/job_local518708628_0012/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local518708628_0012/job_local518708628_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,411 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local518708628_0012/job_local518708628_0012.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,413 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,413 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local518708628_0012
2015-02-10 20:57:07,413 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,414 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:07,414 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,417 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,417 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,418 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,420 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:07,420 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@e26fb32
2015-02-10 20:57:07,421 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:07,422 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:07,564 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:07,564 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:07,564 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:07,565 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:07,565 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:07,566 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,566 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,576 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,576 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:07,576 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:07,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:07,578 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,579 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:07,579 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:07,580 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,581 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:07,581 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 160, total: 590, max: 672
2015-02-10 20:57:07,582 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:07,582 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:07,582 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:07,583 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:07,583 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:07,588 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:07,590 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local518708628_0012_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,592 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:07,592 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local518708628_0012_m_000000_0' done.
2015-02-10 20:57:07,592 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,592 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,593 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:07,593 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local518708628_0012_r_000000_0
2015-02-10 20:57:07,595 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,595 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b14b3af
2015-02-10 20:57:07,596 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:07,597 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local518708628_0012_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:07,599 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#10 about to shuffle output of map attempt_local518708628_0012_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:07,599 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local518708628_0012_m_000000_0
2015-02-10 20:57:07,600 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:07,600 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:07,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,601 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:07,603 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:07,604 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:07,605 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:07,606 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:07,606 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:07,606 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:07,606 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:07,607 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,607 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:07,607 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,616 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2015-02-10 20:57:07,618 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count"]
2015-02-10 20:57:07,630 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local518708628_0012_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,631 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:07,632 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local518708628_0012_r_000000_0 is allowed to commit now
2015-02-10 20:57:07,633 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local518708628_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary/0/task_local518708628_0012_r_000000
2015-02-10 20:57:07,634 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:07,635 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local518708628_0012_r_000000_0' done.
2015-02-10 20:57:07,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local518708628_0012_r_000000_0
2015-02-10 20:57:07,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:07,656 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/count/_temporary
2015-02-10 20:57:07,686 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,717 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,719 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/count
2015-02-10 20:57:07,741 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:07,742 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:07,751 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:07,753 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:07,754 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:07,759 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:07,760 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,760 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 20:57:07,760 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 20:57:07,760 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 20:57:07,760 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 20:57:07,761 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2015-02-10 20:57:07,762 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,763 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:07,789 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:07,807 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:07,823 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local822462648_0013
2015-02-10 20:57:07,841 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity822462648/.staging/job_local822462648_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,841 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity822462648/.staging/job_local822462648_0013/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,891 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local822462648_0013/job_local822462648_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:07,891 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local822462648_0013/job_local822462648_0013.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:07,893 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:07,893 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local822462648_0013
2015-02-10 20:57:07,895 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:07,895 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:07,895 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:07,898 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:07,898 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:07,900 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,902 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:07,902 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6b6a16ae
2015-02-10 20:57:07,903 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:07,903 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:07,958 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:07,958 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:07,958 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:07,959 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:07,959 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:07,960 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:07,961 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:07,970 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:07,971 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:07,971 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:07,974 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:07,975 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,977 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:07,977 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,978 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:07,978 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,979 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:07,979 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 117, total: 624, max: 672
2015-02-10 20:57:07,981 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:07,982 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:07,982 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:07,982 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 20:57:07,983 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:07,987 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:07,989 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local822462648_0013_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:07,992 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:07,992 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local822462648_0013_m_000000_0' done.
2015-02-10 20:57:07,992 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:07,993 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:07,994 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:07,994 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local822462648_0013_r_000000_0
2015-02-10 20:57:07,996 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:07,996 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4d674592
2015-02-10 20:57:07,997 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:07,998 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local822462648_0013_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,001 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#11 about to shuffle output of map attempt_local822462648_0013_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 20:57:08,001 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local822462648_0013_m_000000_0
2015-02-10 20:57:08,001 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 20:57:08,002 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,003 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,004 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,006 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,007 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,008 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,009 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 20:57:08,009 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,010 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,010 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,011 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,012 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,012 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,021 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:08,022 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby"]
2015-02-10 20:57:08,033 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local822462648_0013_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,035 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,035 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local822462648_0013_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,036 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local822462648_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary/0/task_local822462648_0013_r_000000
2015-02-10 20:57:08,038 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,038 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local822462648_0013_r_000000_0' done.
2015-02-10 20:57:08,039 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local822462648_0013_r_000000_0
2015-02-10 20:57:08,039 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,062 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxby/_temporary
2015-02-10 20:57:08,097 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,139 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,141 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxby
2015-02-10 20:57:08,164 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,164 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,173 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,175 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,176 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,181 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min] starting
2015-02-10 20:57:08,181 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,181 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 20:57:08,181 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  parallel execution is enabled: false
2015-02-10 20:57:08,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  starting jobs: 1
2015-02-10 20:57:08,182 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [min]  allocating threads: 1
2015-02-10 20:57:08,182 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2015-02-10 20:57:08,183 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,184 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,208 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,226 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:08,241 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2111875619_0014
2015-02-10 20:57:08,259 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2111875619/.staging/job_local2111875619_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:08,259 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2111875619/.staging/job_local2111875619_0014/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:08,308 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2111875619_0014/job_local2111875619_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:08,308 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2111875619_0014/job_local2111875619_0014.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:08,310 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:08,310 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local2111875619_0014
2015-02-10 20:57:08,311 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2015-02-10 20:57:08,310 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:08,311 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:08,315 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:08,315 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,317 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,319 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:08,319 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14d6c410
2015-02-10 20:57:08,321 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:08,321 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:08,383 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:08,383 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:08,383 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:08,384 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:08,384 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:08,386 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:08,386 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,396 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,397 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:08,397 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:08,399 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:08,399 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,400 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:08,401 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,402 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:08,402 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,403 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:08,403 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 124, total: 674, max: 674
2015-02-10 20:57:08,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:08,405 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:08,405 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:08,406 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 135; bufvoid = 104857600
2015-02-10 20:57:08,406 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:08,410 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:08,412 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2111875619_0014_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,415 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:08,415 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2111875619_0014_m_000000_0' done.
2015-02-10 20:57:08,415 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:08,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:08,416 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2111875619_0014_r_000000_0
2015-02-10 20:57:08,420 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,421 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@55aef45
2015-02-10 20:57:08,422 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=494855776, maxSingleShuffleLimit=123713944, mergeThreshold=326604832, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:08,423 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2111875619_0014_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,425 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#12 about to shuffle output of map attempt_local2111875619_0014_m_000000_0 decomp: 155 len: 159 to MEMORY
2015-02-10 20:57:08,426 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 155 bytes from map-output for attempt_local2111875619_0014_m_000000_0
2015-02-10 20:57:08,426 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 155, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->155
2015-02-10 20:57:08,427 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,428 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,428 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,431 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,431 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,433 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 155 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,433 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 159 bytes from disk
2015-02-10 20:57:08,433 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,433 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,434 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 146 bytes
2015-02-10 20:57:08,434 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,435 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,435 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,444 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2015-02-10 20:57:08,444 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby"]
2015-02-10 20:57:08,455 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2111875619_0014_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,457 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,457 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2111875619_0014_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,458 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2111875619_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary/0/task_local2111875619_0014_r_000000
2015-02-10 20:57:08,459 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,459 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2111875619_0014_r_000000_0' done.
2015-02-10 20:57:08,459 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2111875619_0014_r_000000_0
2015-02-10 20:57:08,459 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,481 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/minby/_temporary
2015-02-10 20:57:08,516 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,552 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,554 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/minby
2015-02-10 20:57:08,587 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,588 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,596 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,598 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,598 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,605 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 20:57:08,605 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,605 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 20:57:08,605 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-10 20:57:08,606 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-10 20:57:08,606 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-10 20:57:08,606 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2015-02-10 20:57:08,607 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,608 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,632 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,650 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:08,665 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1443387747_0015
2015-02-10 20:57:08,683 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1443387747/.staging/job_local1443387747_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:08,683 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1443387747/.staging/job_local1443387747_0015/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:08,730 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1443387747_0015/job_local1443387747_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:08,730 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1443387747_0015/job_local1443387747_0015.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:08,731 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:08,731 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1443387747_0015
2015-02-10 20:57:08,732 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 20:57:08,732 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:08,732 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:08,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:08,735 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,737 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,739 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:08,740 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@13cb8654
2015-02-10 20:57:08,741 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:08,742 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:08,761 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:08,762 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:08,762 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:08,762 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:08,762 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:08,764 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:08,765 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,775 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,775 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:08,776 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:08,777 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:08,777 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,778 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:08,778 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,779 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:08,779 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,780 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:08,780 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 128, total: 676, max: 676
2015-02-10 20:57:08,781 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:08,781 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:08,781 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:08,782 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:08,782 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:08,785 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:08,787 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1443387747_0015_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,789 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:08,789 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1443387747_0015_m_000000_0' done.
2015-02-10 20:57:08,789 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,790 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:08,790 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:08,791 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1443387747_0015_r_000000_0
2015-02-10 20:57:08,792 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:08,792 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3f894e9c
2015-02-10 20:57:08,793 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=496599040, maxSingleShuffleLimit=124149760, mergeThreshold=327755392, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:08,794 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1443387747_0015_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:08,796 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#13 about to shuffle output of map attempt_local1443387747_0015_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:08,797 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1443387747_0015_m_000000_0
2015-02-10 20:57:08,797 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:08,798 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:08,799 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,799 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:08,801 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,801 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:08,803 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:08,803 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:08,803 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:08,803 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:08,804 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:08,804 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,805 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:08,805 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:08,814 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:08,815 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum"]
2015-02-10 20:57:08,826 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1443387747_0015_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:08,827 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:08,828 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1443387747_0015_r_000000_0 is allowed to commit now
2015-02-10 20:57:08,828 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1443387747_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary/0/task_local1443387747_0015_r_000000
2015-02-10 20:57:08,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:08,830 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1443387747_0015_r_000000_0' done.
2015-02-10 20:57:08,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1443387747_0015_r_000000_0
2015-02-10 20:57:08,830 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:08,850 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sum/_temporary
2015-02-10 20:57:08,881 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,914 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:08,916 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sum
2015-02-10 20:57:08,951 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:08,951 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:08,963 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:08,966 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:08,967 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:08,972 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs] starting
2015-02-10 20:57:08,973 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:08,973 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:08,974 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 20:57:08,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  parallel execution is enabled: false
2015-02-10 20:57:08,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  starting jobs: 1
2015-02-10 20:57:08,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum-lhs+sum-rhs]  allocating threads: 1
2015-02-10 20:57:08,976 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2015-02-10 20:57:08,977 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:08,978 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,002 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,003 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,021 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:09,037 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local387217018_0016
2015-02-10 20:57:09,058 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity387217018/.staging/job_local387217018_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,058 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity387217018/.staging/job_local387217018_0016/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,111 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local387217018_0016/job_local387217018_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,112 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local387217018_0016/job_local387217018_0016.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,113 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,114 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local387217018_0016
2015-02-10 20:57:09,114 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:09,114 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,115 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,118 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,118 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,120 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,121 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:09,121 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@30307ae3
2015-02-10 20:57:09,122 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:09,122 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:09,145 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:09,146 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:09,146 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:09,146 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:09,146 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:09,147 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,147 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,159 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:09,160 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,160 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:09,162 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:09,163 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,165 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:09,165 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,167 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:09,167 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,168 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:09,168 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 162, total: 718, max: 718
2015-02-10 20:57:09,171 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,171 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:09,171 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:09,171 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:09,171 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:09,175 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:09,176 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,178 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:09,179 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_m_000000_0' done.
2015-02-10 20:57:09,179 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,179 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,180 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,182 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:09,182 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6f8adce0
2015-02-10 20:57:09,183 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:09,184 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:09,215 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:09,215 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:09,216 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:09,216 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:09,216 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:09,217 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,217 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,228 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:09,228 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,229 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:09,233 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:09,233 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,234 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:09,234 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,235 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:09,235 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,235 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:09,235 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 213, total: 721, max: 721
2015-02-10 20:57:09,237 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,237 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:09,237 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:09,237 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:09,237 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:09,240 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:09,242 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:09,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:09,244 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_m_000001_0' done.
2015-02-10 20:57:09,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:09,245 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local387217018_0016_r_000000_0
2015-02-10 20:57:09,247 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,247 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5b4c1313
2015-02-10 20:57:09,252 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=529491552, maxSingleShuffleLimit=132372888, mergeThreshold=349464448, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:09,253 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local387217018_0016_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:09,255 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local387217018_0016_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:09,256 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local387217018_0016_m_000000_0
2015-02-10 20:57:09,256 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:09,258 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#14 about to shuffle output of map attempt_local387217018_0016_m_000001_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:09,258 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local387217018_0016_m_000001_0
2015-02-10 20:57:09,258 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 2, commitMemory -> 119, usedMemory ->238
2015-02-10 20:57:09,259 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:09,260 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,260 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:09,262 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:09,262 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 220 bytes
2015-02-10 20:57:09,264 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 238 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:09,264 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 240 bytes from disk
2015-02-10 20:57:09,265 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:09,265 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:09,265 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 227 bytes
2015-02-10 20:57:09,266 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,266 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:09,266 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,276 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2015-02-10 20:57:09,277 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum"]
2015-02-10 20:57:09,288 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local387217018_0016_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,289 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:09,289 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local387217018_0016_r_000000_0 is allowed to commit now
2015-02-10 20:57:09,290 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local387217018_0016_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary/0/task_local387217018_0016_r_000000
2015-02-10 20:57:09,291 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:09,291 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local387217018_0016_r_000000_0' done.
2015-02-10 20:57:09,292 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local387217018_0016_r_000000_0
2015-02-10 20:57:09,292 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:09,319 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/mergesum/_temporary
2015-02-10 20:57:09,349 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,379 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,381 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/mergesum
2015-02-10 20:57:09,402 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,402 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,408 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,410 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,411 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,416 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:09,417 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,417 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 20:57:09,417 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:09,417 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:09,417 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:09,418 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2015-02-10 20:57:09,418 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,419 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,443 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,461 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:09,477 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1818488005_0017
2015-02-10 20:57:09,496 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1818488005/.staging/job_local1818488005_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,496 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1818488005/.staging/job_local1818488005_0017/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,549 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1818488005_0017/job_local1818488005_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,549 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1818488005_0017/job_local1818488005_0017.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,551 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,551 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1818488005_0017
2015-02-10 20:57:09,551 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,551 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:09,552 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,555 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,555 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1818488005_0017_m_000000_0
2015-02-10 20:57:09,557 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,559 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:09,559 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7e24f937
2015-02-10 20:57:09,561 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:09,569 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,569 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,575 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,576 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow"]
2015-02-10 20:57:09,579 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,579 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1818488005_0017_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,580 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,580 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1818488005_0017_m_000000_0 is allowed to commit now
2015-02-10 20:57:09,581 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1818488005_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary/0/task_local1818488005_0017_m_000000
2015-02-10 20:57:09,583 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:09,583 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1818488005_0017_m_000000_0' done.
2015-02-10 20:57:09,583 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1818488005_0017_m_000000_0
2015-02-10 20:57:09,583 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,596 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/discardnarrow/_temporary
2015-02-10 20:57:09,630 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,632 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/discardnarrow
2015-02-10 20:57:09,658 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,658 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,667 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,669 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,670 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,674 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:09,674 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,675 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 20:57:09,675 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:09,675 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:09,675 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:09,675 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2015-02-10 20:57:09,676 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,678 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,702 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,722 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:09,751 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1031387534_0018
2015-02-10 20:57:09,770 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1031387534/.staging/job_local1031387534_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,770 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1031387534/.staging/job_local1031387534_0018/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,816 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1031387534_0018/job_local1031387534_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:09,817 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1031387534_0018/job_local1031387534_0018.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:09,818 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:09,818 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1031387534_0018
2015-02-10 20:57:09,818 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:09,818 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:09,819 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:09,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:09,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1031387534_0018_m_000000_0
2015-02-10 20:57:09,823 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:09,824 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:09,824 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6204cfa9
2015-02-10 20:57:09,825 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:09,832 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:09,833 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:09,838 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:09,838 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename"]
2015-02-10 20:57:09,841 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,841 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1031387534_0018_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:09,843 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:09,843 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1031387534_0018_m_000000_0 is allowed to commit now
2015-02-10 20:57:09,843 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1031387534_0018_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary/0/task_local1031387534_0018_m_000000
2015-02-10 20:57:09,844 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:09,845 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1031387534_0018_m_000000_0' done.
2015-02-10 20:57:09,845 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1031387534_0018_m_000000_0
2015-02-10 20:57:09,845 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:09,863 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/rename/_temporary
2015-02-10 20:57:09,892 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,894 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/rename
2015-02-10 20:57:09,917 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:09,918 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:09,932 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:09,935 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:09,935 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:09,942 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:57:09,942 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:09,942 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:09,943 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 20:57:09,943 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 20:57:09,943 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 20:57:09,943 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 20:57:09,944 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2015-02-10 20:57:09,944 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,945 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:09,969 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,970 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:09,989 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:10,005 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1435877350_0019
2015-02-10 20:57:10,022 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1435877350/.staging/job_local1435877350_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,022 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1435877350/.staging/job_local1435877350_0019/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,068 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1435877350_0019/job_local1435877350_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,069 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1435877350_0019/job_local1435877350_0019.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,070 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,070 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1435877350_0019
2015-02-10 20:57:10,070 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:57:10,071 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,071 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,074 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,074 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,076 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,077 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:10,078 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@52129dda
2015-02-10 20:57:10,079 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,079 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,104 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,104 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,104 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,105 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,105 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,108 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,108 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,122 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,122 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,123 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,125 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,125 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,126 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,126 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,127 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,127 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,128 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,128 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 161, total: 750, max: 750
2015-02-10 20:57:10,129 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:10,129 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,129 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,129 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:10,130 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,133 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,135 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:10,137 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_m_000000_0' done.
2015-02-10 20:57:10,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,137 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,139 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,140 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:10,141 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6aed0f19
2015-02-10 20:57:10,143 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,143 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,168 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,168 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,169 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,169 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,169 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,171 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,171 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:10,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,185 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,189 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,190 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,191 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,191 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,192 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,192 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,193 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,193 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 239, total: 750, max: 750
2015-02-10 20:57:10,195 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:10,195 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,195 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,195 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 216; bufvoid = 104857600
2015-02-10 20:57:10,195 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,198 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,200 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:10,202 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:10,202 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_m_000001_0' done.
2015-02-10 20:57:10,202 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,203 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,203 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,204 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1435877350_0019_r_000000_0
2015-02-10 20:57:10,207 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,207 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@240fda77
2015-02-10 20:57:10,208 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551007040, maxSingleShuffleLimit=137751760, mergeThreshold=363664672, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,209 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1435877350_0019_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,211 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1435877350_0019_m_000001_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:10,211 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1435877350_0019_m_000001_0
2015-02-10 20:57:10,211 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->236
2015-02-10 20:57:10,214 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#15 about to shuffle output of map attempt_local1435877350_0019_m_000000_0 decomp: 236 len: 240 to MEMORY
2015-02-10 20:57:10,214 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 236 bytes from map-output for attempt_local1435877350_0019_m_000000_0
2015-02-10 20:57:10,214 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 236, inMemoryMapOutputs.size() -> 2, commitMemory -> 236, usedMemory ->472
2015-02-10 20:57:10,215 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,216 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,216 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,219 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:10,219 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 454 bytes
2015-02-10 20:57:10,221 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 472 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,222 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 474 bytes from disk
2015-02-10 20:57:10,222 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,222 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,222 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 461 bytes
2015-02-10 20:57:10,223 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,224 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,224 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,235 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:10,236 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true"]
2015-02-10 20:57:10,248 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1435877350_0019_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,250 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:10,250 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1435877350_0019_r_000000_0 is allowed to commit now
2015-02-10 20:57:10,250 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1435877350_0019_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary/0/task_local1435877350_0019_r_000000
2015-02-10 20:57:10,252 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:10,252 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1435877350_0019_r_000000_0' done.
2015-02-10 20:57:10,252 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1435877350_0019_r_000000_0
2015-02-10 20:57:10,252 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:10,272 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multimerge+true/_temporary
2015-02-10 20:57:10,301 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,333 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,336 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multimerge+true
2015-02-10 20:57:10,362 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:10,362 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:10,386 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:10,389 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:10,395 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:10,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second] starting
2015-02-10 20:57:10,401 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,402 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 20:57:10,402 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  parallel execution is enabled: false
2015-02-10 20:57:10,402 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  starting jobs: 2
2015-02-10 20:57:10,402 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [second]  allocating threads: 1
2015-02-10 20:57:10,403 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2015-02-10 20:57:10,403 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,404 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,428 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,447 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:10,463 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1115004348_0020
2015-02-10 20:57:10,481 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1115004348/.staging/job_local1115004348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,482 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1115004348/.staging/job_local1115004348_0020/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,533 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1115004348_0020/job_local1115004348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,533 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1115004348_0020/job_local1115004348_0020.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,534 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,535 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,535 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1115004348_0020
2015-02-10 20:57:10,535 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 20:57:10,535 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,537 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,538 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,539 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,540 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:10,541 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4819d373
2015-02-10 20:57:10,542 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,542 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,567 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,568 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,568 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,568 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,568 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,569 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,569 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,579 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:10,579 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2015-02-10 20:57:10,580 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,584 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,584 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,585 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,585 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,586 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:10,586 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,587 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:10,587 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 750, max: 750
2015-02-10 20:57:10,588 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:10,588 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,589 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,589 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:10,589 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:10,592 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,594 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1115004348_0020_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,596 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:10,596 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1115004348_0020_m_000000_0' done.
2015-02-10 20:57:10,597 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,597 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,597 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,598 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1115004348_0020_r_000000_0
2015-02-10 20:57:10,601 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,602 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5c635f9e
2015-02-10 20:57:10,603 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551098752, maxSingleShuffleLimit=137774688, mergeThreshold=363725184, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,604 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1115004348_0020_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,605 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#16 about to shuffle output of map attempt_local1115004348_0020_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:10,606 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local1115004348_0020_m_000000_0
2015-02-10 20:57:10,606 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:10,607 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,609 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,609 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,611 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,611 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:10,612 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,613 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:10,613 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,613 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,614 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:10,614 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,615 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,615 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,623 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2015-02-10 20:57:10,623 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][3708088709/first/]
2015-02-10 20:57:10,634 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1115004348_0020_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,635 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,635 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1115004348_0020_r_000000_0 is allowed to commit now
2015-02-10 20:57:10,636 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1115004348_0020_r_000000_0' to file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/_temporary/0/task_local1115004348_0020_r_000000
2015-02-10 20:57:10,637 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:10,637 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1115004348_0020_r_000000_0' done.
2015-02-10 20:57:10,638 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1115004348_0020_r_000000_0
2015-02-10 20:57:10,638 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:10,659 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2015-02-10 20:57:10,659 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,660 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:10,697 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:10,821 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:10,831 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:10,832 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:10,837 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1035675950_0021
2015-02-10 20:57:10,854 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1035675950/.staging/job_local1035675950_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,854 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1035675950/.staging/job_local1035675950_0021/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,900 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1035675950_0021/job_local1035675950_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:10,901 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1035675950_0021/job_local1035675950_0021.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:10,902 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:10,902 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local1035675950_0021
2015-02-10 20:57:10,902 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2015-02-10 20:57:10,903 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:10,903 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:10,907 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:10,907 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,909 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,910 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/part-00000
2015-02-10 20:57:10,910 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2d22860
2015-02-10 20:57:10,911 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:10,912 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:10,937 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:10,937 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:10,937 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:10,937 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:10,937 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:10,939 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:10,939 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,949 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][3708088709/first/]
2015-02-10 20:57:10,949 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2015-02-10 20:57:10,950 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:10,951 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:10,951 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:10,952 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:10,952 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:10,954 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:10,954 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:10,954 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:10,954 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:10,954 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:10,957 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:10,958 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1035675950_0021_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:10,960 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/3708088709_first_392E7FF95652417AA671A4FC02E969FF/part-00000:0+160
2015-02-10 20:57:10,961 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1035675950_0021_m_000000_0' done.
2015-02-10 20:57:10,961 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,961 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:10,962 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:10,962 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1035675950_0021_r_000000_0
2015-02-10 20:57:10,963 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:10,963 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@66c5c0e5
2015-02-10 20:57:10,964 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551419904, maxSingleShuffleLimit=137854976, mergeThreshold=363937152, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:10,965 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1035675950_0021_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:10,973 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#17 about to shuffle output of map attempt_local1035675950_0021_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:10,973 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1035675950_0021_m_000000_0
2015-02-10 20:57:10,973 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:10,975 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:10,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,977 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:10,979 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,979 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:10,981 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:10,981 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:10,981 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:10,981 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:10,982 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:10,982 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:10,983 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:10,983 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:10,992 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2015-02-10 20:57:10,992 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount"]
2015-02-10 20:57:11,002 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1035675950_0021_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,004 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,004 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1035675950_0021_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,005 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1035675950_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary/0/task_local1035675950_0021_r_000000
2015-02-10 20:57:11,006 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,007 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1035675950_0021_r_000000_0' done.
2015-02-10 20:57:11,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1035675950_0021_r_000000_0
2015-02-10 20:57:11,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,028 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount/_temporary
2015-02-10 20:57:11,060 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,092 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,094 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergecount
2015-02-10 20:57:11,115 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,115 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,124 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,126 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,126 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,131 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 20:57:11,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 20:57:11,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-10 20:57:11,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-10 20:57:11,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-10 20:57:11,134 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2015-02-10 20:57:11,135 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,136 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,160 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,178 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,194 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1241079119_0022
2015-02-10 20:57:11,212 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1241079119/.staging/job_local1241079119_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,212 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1241079119/.staging/job_local1241079119_0022/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,259 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1241079119_0022/job_local1241079119_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,260 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1241079119_0022/job_local1241079119_0022.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,261 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,262 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,262 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,262 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1241079119_0022
2015-02-10 20:57:11,263 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 20:57:11,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,265 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,266 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,268 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,268 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@48ca42a9
2015-02-10 20:57:11,269 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,269 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,294 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,295 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,295 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,295 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,295 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,296 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,296 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,303 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,303 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:11,304 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,305 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,306 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,307 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,307 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,307 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,307 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,308 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,308 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,310 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:11,310 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,310 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,310 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:11,310 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:11,313 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:11,315 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1241079119_0022_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,317 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:11,317 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1241079119_0022_m_000000_0' done.
2015-02-10 20:57:11,317 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,317 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:11,318 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:11,318 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1241079119_0022_r_000000_0
2015-02-10 20:57:11,320 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,320 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@2eb30073
2015-02-10 20:57:11,320 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551419904, maxSingleShuffleLimit=137854976, mergeThreshold=363937152, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:11,322 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1241079119_0022_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:11,323 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#18 about to shuffle output of map attempt_local1241079119_0022_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:11,324 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1241079119_0022_m_000000_0
2015-02-10 20:57:11,324 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:11,325 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:11,326 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,326 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:11,328 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,328 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:11,329 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:11,330 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 20:57:11,330 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:11,330 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,330 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:11,331 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,331 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:11,332 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,339 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:11,339 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average"]
2015-02-10 20:57:11,349 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1241079119_0022_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,351 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,351 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1241079119_0022_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,352 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1241079119_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary/0/task_local1241079119_0022_r_000000
2015-02-10 20:57:11,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,354 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1241079119_0022_r_000000_0' done.
2015-02-10 20:57:11,354 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1241079119_0022_r_000000_0
2015-02-10 20:57:11,355 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,375 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/average/_temporary
2015-02-10 20:57:11,404 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,433 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,436 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/average
2015-02-10 20:57:11,459 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,460 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,467 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,469 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,470 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max] starting
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  parallel execution is enabled: false
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  starting jobs: 1
2015-02-10 20:57:11,476 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [max]  allocating threads: 1
2015-02-10 20:57:11,477 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...splatform/maxbynullsafety
2015-02-10 20:57:11,478 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,479 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,503 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,521 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,536 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local904167718_0023
2015-02-10 20:57:11,553 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity904167718/.staging/job_local904167718_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,553 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity904167718/.staging/job_local904167718_0023/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,599 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local904167718_0023/job_local904167718_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,599 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local904167718_0023/job_local904167718_0023.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,600 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,601 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local904167718_0023
2015-02-10 20:57:11,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,601 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2015-02-10 20:57:11,601 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,604 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,604 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,606 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,607 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,607 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2fbe8a4b
2015-02-10 20:57:11,608 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,608 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,633 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,633 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,634 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,634 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,634 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,635 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,635 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,642 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,642 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:11,643 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,644 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,644 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,645 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,646 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,646 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,646 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,647 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,647 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,649 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:11,649 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,649 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,649 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 98; bufvoid = 104857600
2015-02-10 20:57:11,649 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:11,652 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:11,654 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local904167718_0023_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,656 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:11,656 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local904167718_0023_m_000000_0' done.
2015-02-10 20:57:11,656 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,657 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:11,657 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:11,657 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local904167718_0023_r_000000_0
2015-02-10 20:57:11,659 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,659 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7b8710ed
2015-02-10 20:57:11,660 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551695168, maxSingleShuffleLimit=137923792, mergeThreshold=364118816, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:11,660 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local904167718_0023_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:11,662 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#19 about to shuffle output of map attempt_local904167718_0023_m_000000_0 decomp: 118 len: 122 to MEMORY
2015-02-10 20:57:11,663 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 118 bytes from map-output for attempt_local904167718_0023_m_000000_0
2015-02-10 20:57:11,663 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 118, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->118
2015-02-10 20:57:11,664 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:11,665 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,665 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:11,667 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,667 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 20:57:11,668 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 118 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:11,668 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 122 bytes from disk
2015-02-10 20:57:11,669 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:11,669 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:11,669 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 109 bytes
2015-02-10 20:57:11,670 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,670 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:11,670 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,678 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2015-02-10 20:57:11,679 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety"]
2015-02-10 20:57:11,689 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local904167718_0023_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:11,690 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:11,691 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local904167718_0023_r_000000_0 is allowed to commit now
2015-02-10 20:57:11,691 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local904167718_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary/0/task_local904167718_0023_r_000000
2015-02-10 20:57:11,693 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:11,693 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local904167718_0023_r_000000_0' done.
2015-02-10 20:57:11,693 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local904167718_0023_r_000000_0
2015-02-10 20:57:11,693 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:11,715 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety/_temporary
2015-02-10 20:57:11,744 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,774 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,776 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/maxbynullsafety
2015-02-10 20:57:11,798 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:11,798 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:11,806 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:11,808 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:11,809 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:11,816 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum] starting
2015-02-10 20:57:11,816 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,816 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 20:57:11,816 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  parallel execution is enabled: false
2015-02-10 20:57:11,817 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  starting jobs: 1
2015-02-10 20:57:11,817 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [sum]  allocating threads: 1
2015-02-10 20:57:11,817 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2015-02-10 20:57:11,818 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,819 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:11,844 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:11,862 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:11,877 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1857879576_0024
2015-02-10 20:57:11,894 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1857879576/.staging/job_local1857879576_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,894 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1857879576/.staging/job_local1857879576_0024/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,941 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1857879576_0024/job_local1857879576_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:11,941 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1857879576_0024/job_local1857879576_0024.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:11,942 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:11,942 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1857879576_0024
2015-02-10 20:57:11,943 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2015-02-10 20:57:11,943 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:11,943 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:11,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:11,946 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:11,948 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:11,949 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:11,949 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b3825bf
2015-02-10 20:57:11,950 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:11,951 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:11,976 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:11,977 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:11,977 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:11,977 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:11,977 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:11,978 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:11,978 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:11,985 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:11,985 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:11,986 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:11,995 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:11,995 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,996 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:11,996 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,997 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:11,997 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,997 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:11,998 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 751, max: 751
2015-02-10 20:57:11,999 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:11,999 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:11,999 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:11,999 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 96; bufvoid = 104857600
2015-02-10 20:57:12,000 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:12,003 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,004 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1857879576_0024_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,006 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,007 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1857879576_0024_m_000000_0' done.
2015-02-10 20:57:12,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:12,007 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,008 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1857879576_0024_r_000000_0
2015-02-10 20:57:12,010 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,010 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@52ed6f9
2015-02-10 20:57:12,011 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=551740992, maxSingleShuffleLimit=137935248, mergeThreshold=364149056, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,013 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1857879576_0024_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,015 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#20 about to shuffle output of map attempt_local1857879576_0024_m_000000_0 decomp: 116 len: 120 to MEMORY
2015-02-10 20:57:12,015 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 116 bytes from map-output for attempt_local1857879576_0024_m_000000_0
2015-02-10 20:57:12,016 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 116, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->116
2015-02-10 20:57:12,017 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,018 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,019 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,021 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,021 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 20:57:12,023 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 116 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,024 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 120 bytes from disk
2015-02-10 20:57:12,024 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,024 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,025 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 107 bytes
2015-02-10 20:57:12,026 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,027 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,027 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,033 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2015-02-10 20:57:12,034 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls"]
2015-02-10 20:57:12,043 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1857879576_0024_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,045 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,045 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1857879576_0024_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,046 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1857879576_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary/0/task_local1857879576_0024_r_000000
2015-02-10 20:57:12,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,047 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1857879576_0024_r_000000_0' done.
2015-02-10 20:57:12,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1857879576_0024_r_000000_0
2015-02-10 20:57:12,047 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,066 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/sumnulls/_temporary
2015-02-10 20:57:12,096 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,125 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,127 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/sumnulls
2015-02-10 20:57:12,146 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,147 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,152 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,154 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,154 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,158 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:12,159 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,159 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 20:57:12,159 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 20:57:12,159 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 20:57:12,159 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 20:57:12,160 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2015-02-10 20:57:12,160 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,161 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,185 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,202 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,217 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local308241957_0025
2015-02-10 20:57:12,234 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity308241957/.staging/job_local308241957_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,234 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity308241957/.staging/job_local308241957_0025/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,280 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local308241957_0025/job_local308241957_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,280 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local308241957_0025/job_local308241957_0025.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,281 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,281 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local308241957_0025
2015-02-10 20:57:12,282 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:12,281 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,282 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,285 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,285 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,288 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,289 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:12,289 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6b3a19e3
2015-02-10 20:57:12,291 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:12,291 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:12,317 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:12,317 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:12,317 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:12,318 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:12,318 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:12,319 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,319 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,326 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,327 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:12,327 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:12,330 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:12,330 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:12,330 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:12,331 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 4; bufvoid = 104857600
2015-02-10 20:57:12,331 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 20:57:12,333 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,335 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local308241957_0025_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,337 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,337 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local308241957_0025_m_000000_0' done.
2015-02-10 20:57:12,337 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,337 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,338 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local308241957_0025_r_000000_0
2015-02-10 20:57:12,339 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,340 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5bf43093
2015-02-10 20:57:12,340 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552016256, maxSingleShuffleLimit=138004064, mergeThreshold=364330752, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,341 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local308241957_0025_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,343 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#21 about to shuffle output of map attempt_local308241957_0025_m_000000_0 decomp: 8 len: 12 to MEMORY
2015-02-10 20:57:12,343 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 8 bytes from map-output for attempt_local308241957_0025_m_000000_0
2015-02-10 20:57:12,344 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 8, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->8
2015-02-10 20:57:12,344 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,345 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,346 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,347 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,348 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 20:57:12,349 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 8 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,349 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 12 bytes from disk
2015-02-10 20:57:12,349 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,349 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,350 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 5 bytes
2015-02-10 20:57:12,350 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,351 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,351 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,357 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:12,358 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall"]
2015-02-10 20:57:12,367 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local308241957_0025_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,368 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,368 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local308241957_0025_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,369 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local308241957_0025_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary/0/task_local308241957_0025_r_000000
2015-02-10 20:57:12,370 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,370 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local308241957_0025_r_000000_0' done.
2015-02-10 20:57:12,370 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local308241957_0025_r_000000_0
2015-02-10 20:57:12,370 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,387 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countall/_temporary
2015-02-10 20:57:12,416 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,445 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,447 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countall
2015-02-10 20:57:12,471 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,471 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,480 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,482 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,483 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,489 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name] starting
2015-02-10 20:57:12,490 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,490 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 20:57:12,490 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  parallel execution is enabled: false
2015-02-10 20:57:12,490 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  starting jobs: 1
2015-02-10 20:57:12,490 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [name]  allocating threads: 1
2015-02-10 20:57:12,491 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2015-02-10 20:57:12,491 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,492 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,518 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,536 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,551 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1483521596_0026
2015-02-10 20:57:12,569 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1483521596/.staging/job_local1483521596_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,569 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1483521596/.staging/job_local1483521596_0026/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,617 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1483521596_0026/job_local1483521596_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,617 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1483521596_0026/job_local1483521596_0026.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,618 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,619 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local1483521596_0026
2015-02-10 20:57:12,619 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,619 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2015-02-10 20:57:12,619 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,622 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,622 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,623 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,625 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:12,625 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34985fc6
2015-02-10 20:57:12,626 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:12,626 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:12,652 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:12,652 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:12,652 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:12,652 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:12,653 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:12,654 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,655 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,664 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:12,664 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:12,664 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:12,666 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:12,666 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,667 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:12,668 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,668 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:12,669 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,671 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:12,671 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 240, total: 752, max: 752
2015-02-10 20:57:12,673 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:12,673 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:12,673 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:12,673 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 432; bufvoid = 104857600
2015-02-10 20:57:12,673 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:12,676 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:12,678 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1483521596_0026_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,680 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:12,680 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1483521596_0026_m_000000_0' done.
2015-02-10 20:57:12,680 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,681 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:12,681 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:12,681 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1483521596_0026_r_000000_0
2015-02-10 20:57:12,686 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,686 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@170e3e70
2015-02-10 20:57:12,687 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552108032, maxSingleShuffleLimit=138027008, mergeThreshold=364391328, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:12,688 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1483521596_0026_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:12,691 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#22 about to shuffle output of map attempt_local1483521596_0026_m_000000_0 decomp: 452 len: 456 to MEMORY
2015-02-10 20:57:12,691 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 452 bytes from map-output for attempt_local1483521596_0026_m_000000_0
2015-02-10 20:57:12,692 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 452, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->452
2015-02-10 20:57:12,693 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:12,694 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,694 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:12,696 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,696 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 20:57:12,698 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 452 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:12,698 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 456 bytes from disk
2015-02-10 20:57:12,699 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:12,699 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:12,700 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 436 bytes
2015-02-10 20:57:12,700 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,701 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:12,701 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,710 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2015-02-10 20:57:12,711 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi"]
2015-02-10 20:57:12,722 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1483521596_0026_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,723 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:12,723 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1483521596_0026_r_000000_0 is allowed to commit now
2015-02-10 20:57:12,724 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1483521596_0026_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary/0/task_local1483521596_0026_r_000000
2015-02-10 20:57:12,725 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:12,725 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1483521596_0026_r_000000_0' done.
2015-02-10 20:57:12,725 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1483521596_0026_r_000000_0
2015-02-10 20:57:12,726 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:12,743 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/multi/_temporary
2015-02-10 20:57:12,772 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,804 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,806 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/multi
2015-02-10 20:57:12,825 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:12,825 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:12,830 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:12,831 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:12,832 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:12,837 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:12,838 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:12,838 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 20:57:12,839 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:12,840 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:12,840 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:12,841 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2015-02-10 20:57:12,842 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,843 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:12,867 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:12,884 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:12,899 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1309572996_0027
2015-02-10 20:57:12,917 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1309572996/.staging/job_local1309572996_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,917 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1309572996/.staging/job_local1309572996_0027/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,962 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1309572996_0027/job_local1309572996_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:12,963 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1309572996_0027/job_local1309572996_0027.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:12,964 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:12,965 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:12,965 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:12,966 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1309572996_0027
2015-02-10 20:57:12,966 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:12,968 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:12,968 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1309572996_0027_m_000000_0
2015-02-10 20:57:12,970 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:12,971 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:12,971 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@647be610
2015-02-10 20:57:12,972 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:12,979 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:12,979 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:12,983 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:12,983 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow"]
2015-02-10 20:57:12,986 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:12,986 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1309572996_0027_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:12,987 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:12,988 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1309572996_0027_m_000000_0 is allowed to commit now
2015-02-10 20:57:12,988 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1309572996_0027_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary/0/task_local1309572996_0027_m_000000
2015-02-10 20:57:12,989 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:12,990 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1309572996_0027_m_000000_0' done.
2015-02-10 20:57:12,990 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1309572996_0027_m_000000_0
2015-02-10 20:57:12,990 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,010 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/retainnarrow/_temporary
2015-02-10 20:57:13,039 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,041 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/retainnarrow
2015-02-10 20:57:13,060 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,061 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,066 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,068 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,069 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,073 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:13,073 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,073 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 20:57:13,073 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:13,073 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:13,074 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:13,074 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2015-02-10 20:57:13,075 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,077 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,100 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,118 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,138 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1439863029_0028
2015-02-10 20:57:13,160 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1439863029/.staging/job_local1439863029_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,160 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1439863029/.staging/job_local1439863029_0028/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,219 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1439863029_0028/job_local1439863029_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,219 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1439863029_0028/job_local1439863029_0028.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,221 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,221 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1439863029_0028
2015-02-10 20:57:13,221 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,221 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:13,222 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,225 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,225 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1439863029_0028_m_000000_0
2015-02-10 20:57:13,226 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,228 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:13,228 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5e0469bc
2015-02-10 20:57:13,230 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:13,237 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,237 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,242 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,242 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow"]
2015-02-10 20:57:13,244 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:13,245 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1439863029_0028_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,246 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:13,246 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1439863029_0028_m_000000_0 is allowed to commit now
2015-02-10 20:57:13,247 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1439863029_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary/0/task_local1439863029_0028_m_000000
2015-02-10 20:57:13,248 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:13,248 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1439863029_0028_m_000000_0' done.
2015-02-10 20:57:13,249 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1439863029_0028_m_000000_0
2015-02-10 20:57:13,249 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,265 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/renamenarrow/_temporary
2015-02-10 20:57:13,297 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,299 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/renamenarrow
2015-02-10 20:57:13,325 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,326 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,335 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,337 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,338 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,343 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average] starting
2015-02-10 20:57:13,343 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:13,344 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 20:57:13,344 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  parallel execution is enabled: false
2015-02-10 20:57:13,344 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  starting jobs: 1
2015-02-10 20:57:13,344 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [average]  allocating threads: 1
2015-02-10 20:57:13,345 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2015-02-10 20:57:13,346 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,347 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,371 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,390 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,409 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1491983852_0029
2015-02-10 20:57:13,436 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1491983852/.staging/job_local1491983852_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,436 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1491983852/.staging/job_local1491983852_0029/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,482 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491983852_0029/job_local1491983852_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,483 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1491983852_0029/job_local1491983852_0029.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,484 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,484 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1491983852_0029
2015-02-10 20:57:13,485 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2015-02-10 20:57:13,487 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,487 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,488 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,489 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:13,490 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4876d42
2015-02-10 20:57:13,491 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:13,491 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:13,508 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:13,508 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:13,508 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:13,509 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:13,509 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:13,510 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,510 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,517 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:13,518 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:13,518 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:13,522 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:13,522 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,523 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:13,523 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,526 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:13,526 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,527 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:13,527 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 228, total: 752, max: 752
2015-02-10 20:57:13,528 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:13,528 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:13,528 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:13,528 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 180; bufvoid = 104857600
2015-02-10 20:57:13,529 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:13,531 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:13,533 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1491983852_0029_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,535 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:13,535 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491983852_0029_m_000000_0' done.
2015-02-10 20:57:13,535 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,535 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,536 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:13,536 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1491983852_0029_r_000000_0
2015-02-10 20:57:13,538 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,538 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@b920c51
2015-02-10 20:57:13,538 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552337408, maxSingleShuffleLimit=138084352, mergeThreshold=364542688, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:13,539 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1491983852_0029_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:13,541 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#23 about to shuffle output of map attempt_local1491983852_0029_m_000000_0 decomp: 200 len: 204 to MEMORY
2015-02-10 20:57:13,541 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 200 bytes from map-output for attempt_local1491983852_0029_m_000000_0
2015-02-10 20:57:13,541 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 200, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->200
2015-02-10 20:57:13,542 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:13,543 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,543 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:13,545 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:13,545 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:13,546 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 200 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:13,547 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 204 bytes from disk
2015-02-10 20:57:13,547 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:13,547 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:13,547 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 191 bytes
2015-02-10 20:57:13,548 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,548 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:13,548 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,556 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2015-02-10 20:57:13,556 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull"]
2015-02-10 20:57:13,567 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1491983852_0029_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,568 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:13,568 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1491983852_0029_r_000000_0 is allowed to commit now
2015-02-10 20:57:13,569 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1491983852_0029_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary/0/task_local1491983852_0029_r_000000
2015-02-10 20:57:13,570 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:13,570 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1491983852_0029_r_000000_0' done.
2015-02-10 20:57:13,570 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1491983852_0029_r_000000_0
2015-02-10 20:57:13,570 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:13,591 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/averagenull/_temporary
2015-02-10 20:57:13,621 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,653 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,655 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/averagenull
2015-02-10 20:57:13,672 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,673 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,677 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,679 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,679 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,683 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce] starting
2015-02-10 20:57:13,683 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,684 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 20:57:13,684 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  parallel execution is enabled: false
2015-02-10 20:57:13,684 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  starting jobs: 1
2015-02-10 20:57:13,685 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [coerce]  allocating threads: 1
2015-02-10 20:57:13,686 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2015-02-10 20:57:13,687 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,688 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,712 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,730 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,745 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local314091455_0030
2015-02-10 20:57:13,762 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity314091455/.staging/job_local314091455_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,762 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity314091455/.staging/job_local314091455_0030/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,816 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local314091455_0030/job_local314091455_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,817 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local314091455_0030/job_local314091455_0030.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:13,818 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:13,818 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:13,819 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:13,818 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local314091455_0030
2015-02-10 20:57:13,819 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2015-02-10 20:57:13,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:13,821 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local314091455_0030_m_000000_0
2015-02-10 20:57:13,822 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:13,824 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt
2015-02-10 20:57:13,824 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3512106c
2015-02-10 20:57:13,825 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-10 20:57:13,831 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:13,832 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:13,835 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt"]
2015-02-10 20:57:13,836 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce"]
2015-02-10 20:57:13,838 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:13,838 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local314091455_0030_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:13,840 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:13,840 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local314091455_0030_m_000000_0 is allowed to commit now
2015-02-10 20:57:13,841 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local314091455_0030_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary/0/task_local314091455_0030_m_000000
2015-02-10 20:57:13,842 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lower.txt:0+19
2015-02-10 20:57:13,842 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local314091455_0030_m_000000_0' done.
2015-02-10 20:57:13,842 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local314091455_0030_m_000000_0
2015-02-10 20:57:13,842 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:13,852 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/coerce/_temporary
2015-02-10 20:57:13,883 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,885 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/coerce
2015-02-10 20:57:13,907 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:13,907 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:13,914 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:13,916 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:13,916 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:13,921 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first] starting
2015-02-10 20:57:13,921 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 20:57:13,921 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 20:57:13,922 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  parallel execution is enabled: false
2015-02-10 20:57:13,922 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  starting jobs: 1
2015-02-10 20:57:13,922 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [first]  allocating threads: 1
2015-02-10 20:57:13,922 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...tnfieldswithoutcomparator
2015-02-10 20:57:13,923 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,924 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:13,946 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:13,965 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:13,982 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local715569249_0031
2015-02-10 20:57:13,998 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity715569249/.staging/job_local715569249_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:13,998 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity715569249/.staging/job_local715569249_0031/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,046 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local715569249_0031/job_local715569249_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:14,047 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local715569249_0031/job_local715569249_0031.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,048 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,049 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,049 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local715569249_0031
2015-02-10 20:57:14,049 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2015-02-10 20:57:14,049 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,051 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,052 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,053 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,055 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt
2015-02-10 20:57:14,055 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6600167a
2015-02-10 20:57:14,056 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,056 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,087 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,088 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,088 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,088 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,088 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,089 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,089 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,095 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt"]
2015-02-10 20:57:14,095 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:14,096 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,098 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,098 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,099 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,099 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,101 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:14,101 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,102 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,102 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 20:57:14,102 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:14,104 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,106 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local715569249_0031_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,109 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs+rhs-cross-rev.txt:0+221
2015-02-10 20:57:14,109 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local715569249_0031_m_000000_0' done.
2015-02-10 20:57:14,109 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,109 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,110 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,110 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local715569249_0031_r_000000_0
2015-02-10 20:57:14,112 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,112 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@573c3ba2
2015-02-10 20:57:14,113 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552383296, maxSingleShuffleLimit=138095824, mergeThreshold=364572992, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,114 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local715569249_0031_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,117 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#24 about to shuffle output of map attempt_local715569249_0031_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 20:57:14,117 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local715569249_0031_m_000000_0
2015-02-10 20:57:14,117 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->117
2015-02-10 20:57:14,118 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,119 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,119 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,122 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,122 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 20:57:14,124 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 117 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,124 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 121 bytes from disk
2015-02-10 20:57:14,125 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,125 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,126 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 108 bytes
2015-02-10 20:57:14,127 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,127 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,128 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,133 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2015-02-10 20:57:14,133 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator"]
2015-02-10 20:57:14,143 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local715569249_0031_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,144 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,144 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local715569249_0031_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,145 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local715569249_0031_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary/0/task_local715569249_0031_r_000000
2015-02-10 20:57:14,146 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,147 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local715569249_0031_r_000000_0' done.
2015-02-10 20:57:14,147 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local715569249_0031_r_000000_0
2015-02-10 20:57:14,147 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:14,166 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator/_temporary
2015-02-10 20:57:14,194 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,196 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/firstnfieldswithoutcomparator
2015-02-10 20:57:14,219 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:14,220 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:14,245 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:14,247 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:14,269 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:14,274 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...] starting
2015-02-10 20:57:14,274 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:14,274 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 20:57:14,275 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  parallel execution is enabled: false
2015-02-10 20:57:14,275 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  starting jobs: 5
2015-02-10 20:57:14,275 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count-lhs*source*count...]  allocating threads: 1
2015-02-10 20:57:14,275 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2015-02-10 20:57:14,276 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,278 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,301 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,319 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:14,334 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2013585284_0032
2015-02-10 20:57:14,350 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2013585284/.staging/job_local2013585284_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:14,350 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity2013585284/.staging/job_local2013585284_0032/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,395 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2013585284_0032/job_local2013585284_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:14,395 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2013585284_0032/job_local2013585284_0032.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,396 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,397 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local2013585284_0032
2015-02-10 20:57:14,397 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,397 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:14,397 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,399 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,401 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,402 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:14,402 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@499aed36
2015-02-10 20:57:14,403 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,403 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,429 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,429 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,429 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,429 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,429 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,431 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,431 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,439 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:14,439 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2015-02-10 20:57:14,439 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,440 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,441 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 5, with threshold: 2
2015-02-10 20:57:14,442 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,443 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 7, with threshold: 2
2015-02-10 20:57:14,443 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 241, total: 752, max: 752
2015-02-10 20:57:14,444 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:14,444 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,444 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,445 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 99; bufvoid = 104857600
2015-02-10 20:57:14,445 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600
2015-02-10 20:57:14,448 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,449 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2013585284_0032_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,451 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:14,451 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2013585284_0032_m_000000_0' done.
2015-02-10 20:57:14,451 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,452 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2013585284_0032_r_000000_0
2015-02-10 20:57:14,456 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,456 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5482fb8f
2015-02-10 20:57:14,457 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=552566784, maxSingleShuffleLimit=138141696, mergeThreshold=364694080, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,458 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2013585284_0032_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,460 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#25 about to shuffle output of map attempt_local2013585284_0032_m_000000_0 decomp: 119 len: 123 to MEMORY
2015-02-10 20:57:14,460 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 119 bytes from map-output for attempt_local2013585284_0032_m_000000_0
2015-02-10 20:57:14,460 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 119, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->119
2015-02-10 20:57:14,461 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,462 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,462 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,464 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,465 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:14,466 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 119 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,466 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 123 bytes from disk
2015-02-10 20:57:14,467 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,467 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,468 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 110 bytes
2015-02-10 20:57:14,468 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,469 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,469 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,474 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2015-02-10 20:57:14,474 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:14,483 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local2013585284_0032_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,485 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local2013585284_0032_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,485 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2013585284_0032_r_000000_0' to file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/_temporary/0/task_local2013585284_0032_r_000000
2015-02-10 20:57:14,486 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,486 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local2013585284_0032_r_000000_0' done.
2015-02-10 20:57:14,487 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2013585284_0032_r_000000_0
2015-02-10 20:57:14,487 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:14,507 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2015-02-10 20:57:14,507 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,508 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:14,543 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:14,560 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:14,571 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:14,571 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:14,576 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1416992176_0033
2015-02-10 20:57:14,593 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1416992176/.staging/job_local1416992176_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:14,593 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1416992176/.staging/job_local1416992176_0033/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,640 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1416992176_0033/job_local1416992176_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:14,641 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1416992176_0033/job_local1416992176_0033.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:14,642 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:14,642 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:14,643 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:14,643 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1416992176_0033
2015-02-10 20:57:14,643 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:14,646 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:14,646 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,647 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,648 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:14,649 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ee55d17
2015-02-10 20:57:14,650 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:14,650 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:14,687 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:14,687 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:14,687 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:14,688 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:14,688 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:14,689 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:14,689 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,695 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:14,695 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 20:57:14,695 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:14,696 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:14,696 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 96, total: 752, max: 752
2015-02-10 20:57:14,697 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:14,697 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 96, total: 752, max: 752
2015-02-10 20:57:14,698 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:14,698 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:14,698 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:14,699 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:14,958 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:14,961 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:14,963 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1416992176_0033_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,965 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:14,965 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1416992176_0033_m_000000_0' done.
2015-02-10 20:57:14,965 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,965 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:14,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:14,966 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1416992176_0033_r_000000_0
2015-02-10 20:57:14,967 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:14,968 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6cd8739a
2015-02-10 20:57:14,968 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:14,969 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1416992176_0033_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:14,970 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#26 about to shuffle output of map attempt_local1416992176_0033_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:14,971 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local1416992176_0033_m_000000_0
2015-02-10 20:57:14,971 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:14,972 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:14,973 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,973 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:14,975 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,975 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:14,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:14,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:14,976 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:14,977 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:14,977 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:14,977 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,978 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:14,978 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:14,983 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2015-02-10 20:57:14,983 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][662217636/count-lhs/]
2015-02-10 20:57:14,992 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1416992176_0033_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:14,993 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:14,993 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1416992176_0033_r_000000_0 is allowed to commit now
2015-02-10 20:57:14,994 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1416992176_0033_r_000000_0' to file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/_temporary/0/task_local1416992176_0033_r_000000
2015-02-10 20:57:14,995 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:14,995 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1416992176_0033_r_000000_0' done.
2015-02-10 20:57:14,995 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1416992176_0033_r_000000_0
2015-02-10 20:57:14,995 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,013 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2015-02-10 20:57:15,014 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,015 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,050 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,068 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:15,078 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,078 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,084 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local892925828_0034
2015-02-10 20:57:15,100 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity892925828/.staging/job_local892925828_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,100 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity892925828/.staging/job_local892925828_0034/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,145 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local892925828_0034/job_local892925828_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,145 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local892925828_0034/job_local892925828_0034.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,146 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,147 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local892925828_0034
2015-02-10 20:57:15,147 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,147 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,148 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,150 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,151 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,152 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,154 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:15,154 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@439ca904
2015-02-10 20:57:15,156 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,156 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,175 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,175 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,175 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,175 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,175 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,177 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,177 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:15,185 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 20:57:15,185 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:15,187 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-10 20:57:15,187 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 172, total: 427, max: 672
2015-02-10 20:57:15,187 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:15,187 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-10 20:57:15,188 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 172, total: 427, max: 672
2015-02-10 20:57:15,188 INFO  assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-10 20:57:15,189 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:15,189 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,189 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,189 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 55; bufvoid = 104857600
2015-02-10 20:57:15,189 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,192 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,194 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local892925828_0034_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,196 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:15,196 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local892925828_0034_m_000000_0' done.
2015-02-10 20:57:15,196 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,196 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,197 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,197 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local892925828_0034_r_000000_0
2015-02-10 20:57:15,199 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,199 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@6d0cad7b
2015-02-10 20:57:15,200 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,200 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local892925828_0034_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,204 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#27 about to shuffle output of map attempt_local892925828_0034_m_000000_0 decomp: 67 len: 71 to MEMORY
2015-02-10 20:57:15,204 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 67 bytes from map-output for attempt_local892925828_0034_m_000000_0
2015-02-10 20:57:15,205 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 67, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->67
2015-02-10 20:57:15,205 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,206 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,207 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,209 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,209 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:15,210 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 67 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,211 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 71 bytes from disk
2015-02-10 20:57:15,211 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,211 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,211 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 58 bytes
2015-02-10 20:57:15,212 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,212 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,212 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,218 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2015-02-10 20:57:15,218 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][511480183/count-rhs/]
2015-02-10 20:57:15,227 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local892925828_0034_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,228 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:15,228 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local892925828_0034_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,229 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local892925828_0034_r_000000_0' to file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/_temporary/0/task_local892925828_0034_r_000000
2015-02-10 20:57:15,230 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,230 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local892925828_0034_r_000000_0' done.
2015-02-10 20:57:15,230 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local892925828_0034_r_000000_0
2015-02-10 20:57:15,230 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,243 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2015-02-10 20:57:15,243 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,244 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,279 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,292 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,310 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:15,320 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,320 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,326 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1058891099_0035
2015-02-10 20:57:15,343 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1058891099/.staging/job_local1058891099_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,343 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1058891099/.staging/job_local1058891099_0035/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,389 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1058891099_0035/job_local1058891099_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,390 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1058891099_0035/job_local1058891099_0035.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,391 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,392 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1058891099_0035
2015-02-10 20:57:15,392 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,392 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,392 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,395 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,395 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,397 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,398 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000
2015-02-10 20:57:15,398 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ff9ddb5
2015-02-10 20:57:15,400 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,400 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,423 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,423 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,423 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,423 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,423 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,426 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,426 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,433 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][4782051044/source/]
2015-02-10 20:57:15,433 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,436 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:15,436 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,436 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,436 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,436 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,439 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,441 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,442 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/4782051044_source_FD52B13C054D45C88E578801EBB52573/part-00000:0+160
2015-02-10 20:57:15,442 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_m_000000_0' done.
2015-02-10 20:57:15,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,443 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,444 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,445 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/part-00000
2015-02-10 20:57:15,445 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7809ead8
2015-02-10 20:57:15,447 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,447 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,464 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,464 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,465 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,465 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,465 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,466 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,466 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,474 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][511480183/count-rhs/]
2015-02-10 20:57:15,475 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,476 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:15,477 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,477 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,477 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,477 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,479 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,481 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:15,483 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/511480183_count_rhs_2F723E24535B40D9B8C83AABD3957A53/part-00000:0+160
2015-02-10 20:57:15,483 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_m_000001_0' done.
2015-02-10 20:57:15,483 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,483 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,484 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1058891099_0035_r_000000_0
2015-02-10 20:57:15,485 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,485 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@42793673
2015-02-10 20:57:15,486 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,487 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1058891099_0035_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,488 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1058891099_0035_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,489 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1058891099_0035_m_000001_0
2015-02-10 20:57:15,489 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 20:57:15,491 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#28 about to shuffle output of map attempt_local1058891099_0035_m_000000_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,492 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1058891099_0035_m_000000_0
2015-02-10 20:57:15,492 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->154
2015-02-10 20:57:15,493 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,495 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,496 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,498 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:15,498 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 134 bytes
2015-02-10 20:57:15,500 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 154 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,500 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 156 bytes from disk
2015-02-10 20:57:15,501 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,501 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,501 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 142 bytes
2015-02-10 20:57:15,502 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,503 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,503 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,508 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2015-02-10 20:57:15,509 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5223282736/source_count-rhs/]
2015-02-10 20:57:15,523 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,523 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,530 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1058891099_0035_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,531 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,531 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1058891099_0035_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,532 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1058891099_0035_r_000000_0' to file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/_temporary/0/task_local1058891099_0035_r_000000
2015-02-10 20:57:15,533 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,533 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1058891099_0035_r_000000_0' done.
2015-02-10 20:57:15,533 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1058891099_0035_r_000000_0
2015-02-10 20:57:15,534 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,552 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2015-02-10 20:57:15,552 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,553 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:15,588 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,602 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:15,620 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:15,629 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:15,629 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:15,635 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1092641491_0036
2015-02-10 20:57:15,651 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1092641491/.staging/job_local1092641491_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,651 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1092641491/.staging/job_local1092641491_0036/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,704 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1092641491_0036/job_local1092641491_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:15,704 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1092641491_0036/job_local1092641491_0036.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:15,706 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:15,706 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1092641491_0036
2015-02-10 20:57:15,706 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2015-02-10 20:57:15,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:15,707 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:15,834 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:15,834 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,836 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,838 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/part-00000
2015-02-10 20:57:15,838 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1f78a4e8
2015-02-10 20:57:15,840 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,840 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,858 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,858 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,858 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,858 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,859 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,861 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,861 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,868 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5223282736/source_count-rhs/]
2015-02-10 20:57:15,869 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,870 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:15,871 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,871 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,871 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 105; bufvoid = 104857600
2015-02-10 20:57:15,871 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,874 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,875 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,877 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/5223282736_source_count_rhs_0EFDC0F8252B47848ABB1557CC1685D9/part-00000:0+200
2015-02-10 20:57:15,877 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_m_000000_0' done.
2015-02-10 20:57:15,877 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,877 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,878 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,879 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/part-00000
2015-02-10 20:57:15,880 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3ccc2187
2015-02-10 20:57:15,881 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:15,881 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:15,899 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:15,899 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:15,899 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:15,899 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:15,899 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:15,900 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:15,901 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,911 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][662217636/count-lhs/]
2015-02-10 20:57:15,911 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,914 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:15,914 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:15,915 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:15,915 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 65; bufvoid = 104857600
2015-02-10 20:57:15,915 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:15,918 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:15,920 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:15,921 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/tmp/hadoop-teamcity/662217636_count_lhs_4FB7D5D97F0A45A2B1A5E4C0AC039061/part-00000:0+160
2015-02-10 20:57:15,922 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_m_000001_0' done.
2015-02-10 20:57:15,922 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,922 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:15,923 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:15,923 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1092641491_0036_r_000000_0
2015-02-10 20:57:15,925 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:15,925 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5f92e49f
2015-02-10 20:57:15,926 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:15,926 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1092641491_0036_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:15,928 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1092641491_0036_m_000001_0 decomp: 77 len: 81 to MEMORY
2015-02-10 20:57:15,928 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 77 bytes from map-output for attempt_local1092641491_0036_m_000001_0
2015-02-10 20:57:15,928 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 77, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->77
2015-02-10 20:57:15,930 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#29 about to shuffle output of map attempt_local1092641491_0036_m_000000_0 decomp: 117 len: 121 to MEMORY
2015-02-10 20:57:15,931 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 117 bytes from map-output for attempt_local1092641491_0036_m_000000_0
2015-02-10 20:57:15,931 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 117, inMemoryMapOutputs.size() -> 2, commitMemory -> 77, usedMemory ->194
2015-02-10 20:57:15,931 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:15,932 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,932 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:15,934 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:15,934 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2015-02-10 20:57:15,936 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 194 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:15,936 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 196 bytes from disk
2015-02-10 20:57:15,936 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:15,936 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:15,937 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2015-02-10 20:57:15,937 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,938 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:15,938 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:15,944 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2015-02-10 20:57:15,945 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway"]
2015-02-10 20:57:15,953 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,953 INFO  collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2015-02-10 20:57:15,955 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1092641491_0036_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:15,956 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:15,957 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1092641491_0036_r_000000_0 is allowed to commit now
2015-02-10 20:57:15,957 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1092641491_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary/0/task_local1092641491_0036_r_000000
2015-02-10 20:57:15,958 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:15,958 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1092641491_0036_r_000000_0' done.
2015-02-10 20:57:15,958 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1092641491_0036_r_000000_0
2015-02-10 20:57:15,959 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:15,978 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2015-02-10 20:57:16,006 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,035 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,036 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/samesourcemergethreeway
2015-02-10 20:57:16,058 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,059 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,066 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,069 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,069 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,077 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs] starting
2015-02-10 20:57:16,077 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:16,077 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,078 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 20:57:16,078 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  parallel execution is enabled: false
2015-02-10 20:57:16,078 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  starting jobs: 1
2015-02-10 20:57:16,078 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [lhs+rhs]  allocating threads: 1
2015-02-10 20:57:16,079 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2015-02-10 20:57:16,079 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,080 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,109 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,109 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,129 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:2
2015-02-10 20:57:16,144 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local533979663_0037
2015-02-10 20:57:16,162 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity533979663/.staging/job_local533979663_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,163 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity533979663/.staging/job_local533979663_0037/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,208 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local533979663_0037/job_local533979663_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,208 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local533979663_0037/job_local533979663_0037.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,209 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,210 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local533979663_0037
2015-02-10 20:57:16,210 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,210 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,210 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2015-02-10 20:57:16,212 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,213 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,214 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,215 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,215 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3026a5f8
2015-02-10 20:57:16,216 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,217 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,234 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,234 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,234 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,234 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,234 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,235 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,235 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,241 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,241 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,247 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:16,247 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,247 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,247 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,248 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,250 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,252 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,253 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,253 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_m_000000_0' done.
2015-02-10 20:57:16,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,254 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,255 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,256 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt
2015-02-10 20:57:16,256 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@27d2b7b3
2015-02-10 20:57:16,257 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,258 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,275 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,275 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,275 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,275 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,275 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,279 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,279 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,284 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt"]
2015-02-10 20:57:16,284 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,287 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:16,287 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,287 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,287 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,288 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,290 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,292 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_m_000001_0 is done. And is in the process of committing
2015-02-10 20:57:16,293 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/rhs.txt:0+51
2015-02-10 20:57:16,293 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_m_000001_0' done.
2015-02-10 20:57:16,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,294 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local533979663_0037_r_000000_0
2015-02-10 20:57:16,296 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,296 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d3313b1
2015-02-10 20:57:16,297 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,299 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local533979663_0037_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,300 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local533979663_0037_m_000001_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,301 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local533979663_0037_m_000001_0
2015-02-10 20:57:16,301 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:16,303 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#30 about to shuffle output of map attempt_local533979663_0037_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,303 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local533979663_0037_m_000000_0
2015-02-10 20:57:16,303 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 2, commitMemory -> 87, usedMemory ->174
2015-02-10 20:57:16,304 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,305 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,305 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 2 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,307 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 2 sorted segments
2015-02-10 20:57:16,307 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 2 segments left of total size: 156 bytes
2015-02-10 20:57:16,308 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 2 segments, 174 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,309 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 176 bytes from disk
2015-02-10 20:57:16,309 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,309 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,309 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 163 bytes
2015-02-10 20:57:16,310 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,310 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,310 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,316 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2015-02-10 20:57:16,316 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2015-02-10 20:57:16,326 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local533979663_0037_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,327 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 2 / 2 copied.
2015-02-10 20:57:16,327 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local533979663_0037_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,328 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local533979663_0037_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary/0/task_local533979663_0037_r_000000
2015-02-10 20:57:16,329 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,329 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local533979663_0037_r_000000_0' done.
2015-02-10 20:57:16,329 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local533979663_0037_r_000000_0
2015-02-10 20:57:16,329 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,358 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2015-02-10 20:57:16,387 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,389 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/uniquemerge-nondeterministic
2015-02-10 20:57:16,407 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,408 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,414 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,416 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,417 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,421 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape] starting
2015-02-10 20:57:16,421 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,422 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 20:57:16,422 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  parallel execution is enabled: false
2015-02-10 20:57:16,422 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  starting jobs: 1
2015-02-10 20:57:16,423 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [shape]  allocating threads: 1
2015-02-10 20:57:16,425 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2015-02-10 20:57:16,425 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,427 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,449 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,467 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:16,482 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1602462514_0038
2015-02-10 20:57:16,499 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1602462514/.staging/job_local1602462514_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,499 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity1602462514/.staging/job_local1602462514_0038/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,545 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1602462514_0038/job_local1602462514_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,545 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1602462514_0038/job_local1602462514_0038.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,547 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,547 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,547 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,547 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1602462514_0038
2015-02-10 20:57:16,548 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2015-02-10 20:57:16,550 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,550 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,551 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,553 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,553 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5db533b0
2015-02-10 20:57:16,554 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,554 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,571 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,571 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,572 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,572 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,572 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,573 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,573 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,578 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,578 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 20:57:16,581 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:16,581 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,581 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,581 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2015-02-10 20:57:16,581 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2015-02-10 20:57:16,584 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,587 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1602462514_0038_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,589 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,589 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1602462514_0038_m_000000_0' done.
2015-02-10 20:57:16,589 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,589 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,590 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,590 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local1602462514_0038_r_000000_0
2015-02-10 20:57:16,591 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,592 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1d489e0e
2015-02-10 20:57:16,592 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,593 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local1602462514_0038_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,594 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#31 about to shuffle output of map attempt_local1602462514_0038_m_000000_0 decomp: 87 len: 91 to MEMORY
2015-02-10 20:57:16,595 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 87 bytes from map-output for attempt_local1602462514_0038_m_000000_0
2015-02-10 20:57:16,595 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 87, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->87
2015-02-10 20:57:16,596 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,597 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,597 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,599 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,599 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:16,600 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 87 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,601 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 91 bytes from disk
2015-02-10 20:57:16,601 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,601 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,601 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2015-02-10 20:57:16,602 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,602 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,602 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,606 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2015-02-10 20:57:16,606 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique"]
2015-02-10 20:57:16,615 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local1602462514_0038_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,616 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,616 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local1602462514_0038_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,617 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1602462514_0038_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary/0/task_local1602462514_0038_r_000000
2015-02-10 20:57:16,618 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,618 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local1602462514_0038_r_000000_0' done.
2015-02-10 20:57:16,618 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local1602462514_0038_r_000000_0
2015-02-10 20:57:16,619 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,634 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/unique/_temporary
2015-02-10 20:57:16,662 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,664 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/unique
2015-02-10 20:57:16,684 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-10 20:57:16,684 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-10 20:57:16,690 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-10 20:57:16,692 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-10 20:57:16,693 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(996)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-10 20:57:16,697 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count] starting
2015-02-10 20:57:16,697 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,697 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 20:57:16,697 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  parallel execution is enabled: false
2015-02-10 20:57:16,698 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  starting jobs: 1
2015-02-10 20:57:16,698 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [count]  allocating threads: 1
2015-02-10 20:57:16,698 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2015-02-10 20:57:16,699 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,700 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-10 20:57:16,723 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,741 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-10 20:57:16,755 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local195034871_0039
2015-02-10 20:57:16,773 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity195034871/.staging/job_local195034871_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,773 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/tmp/cascading/staging/teamcity195034871/.staging/job_local195034871_0039/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,821 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local195034871_0039/job_local195034871_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-10 20:57:16,821 WARN  conf.Configuration (Configuration.java:loadProperty(2345)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local195034871_0039/job_local195034871_0039.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-10 20:57:16,823 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-10 20:57:16,823 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local195034871_0039
2015-02-10 20:57:16,824 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-10 20:57:16,824 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2015-02-10 20:57:16,824 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-10 20:57:16,826 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-10 20:57:16,827 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,828 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,829 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt
2015-02-10 20:57:16,829 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@78f278dd
2015-02-10 20:57:16,830 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-10 20:57:16,830 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-10 20:57:16,853 INFO  mapred.MapTask (MapTask.java:setEquator(1181)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-10 20:57:16,853 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-10 20:57:16,853 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-10 20:57:16,853 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-10 20:57:16,854 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-10 20:57:16,855 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-10 20:57:16,855 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,863 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt"]
2015-02-10 20:57:16,863 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:16,864 INFO  assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-10 20:57:16,870 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-10 20:57:16,870 INFO  mapred.MapTask (MapTask.java:flush(1435)) - Starting flush of map output
2015-02-10 20:57:16,870 INFO  mapred.MapTask (MapTask.java:flush(1453)) - Spilling map output
2015-02-10 20:57:16,871 INFO  mapred.MapTask (MapTask.java:flush(1454)) - bufstart = 0; bufend = 6; bufvoid = 104857600
2015-02-10 20:57:16,871 INFO  mapred.MapTask (MapTask.java:flush(1456)) - kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600
2015-02-10 20:57:16,873 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1639)) - Finished spill 0
2015-02-10 20:57:16,875 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local195034871_0039_m_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,877 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/classes/test/data/lhs.txt:0+52
2015-02-10 20:57:16,877 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local195034871_0039_m_000000_0' done.
2015-02-10 20:57:16,878 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,878 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-10 20:57:16,878 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-10 20:57:16,879 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local195034871_0039_r_000000_0
2015-02-10 20:57:16,880 INFO  mapred.Task (Task.java:initialize(581)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-10 20:57:16,881 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@c7d9406
2015-02-10 20:57:16,881 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-10 20:57:16,882 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local195034871_0039_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-10 20:57:16,884 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#32 about to shuffle output of map attempt_local195034871_0039_m_000000_0 decomp: 10 len: 14 to MEMORY
2015-02-10 20:57:16,884 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 10 bytes from map-output for attempt_local195034871_0039_m_000000_0
2015-02-10 20:57:16,884 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 10, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->10
2015-02-10 20:57:16,885 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-10 20:57:16,886 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,886 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-10 20:57:16,888 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,888 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 20:57:16,890 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 10 bytes to disk to satisfy reduce memory limit
2015-02-10 20:57:16,890 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 14 bytes from disk
2015-02-10 20:57:16,891 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-10 20:57:16,891 INFO  mapred.Merger (Merger.java:merge(589)) - Merging 1 sorted segments
2015-02-10 20:57:16,891 INFO  mapred.Merger (Merger.java:merge(688)) - Down to the last merge-pass, with 1 segments left of total size: 7 bytes
2015-02-10 20:57:16,892 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,893 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-10 20:57:16,893 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-10 20:57:16,901 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2015-02-10 20:57:16,901 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull"]
2015-02-10 20:57:16,909 INFO  mapred.Task (Task.java:done(995)) - Task:attempt_local195034871_0039_r_000000_0 is done. And is in the process of committing
2015-02-10 20:57:16,910 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-10 20:57:16,911 INFO  mapred.Task (Task.java:commit(1156)) - Task attempt_local195034871_0039_r_000000_0 is allowed to commit now
2015-02-10 20:57:16,911 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local195034871_0039_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary/0/task_local195034871_0039_r_000000
2015-02-10 20:57:16,912 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-10 20:57:16,912 INFO  mapred.Task (Task.java:sendDone(1115)) - Task 'attempt_local195034871_0039_r_000000_0' done.
2015-02-10 20:57:16,912 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local195034871_0039_r_000000_0
2015-02-10 20:57:16,913 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-10 20:57:16,932 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output/hadoop2-mr1/assemblyhelpersplatform/countnullnotnull/_temporary
2015-02-10 20:57:16,961 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,989 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(253)) - Total input paths to process : 1
2015-02-10 20:57:16,991 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.3.x/build/test/output//hadoop2-mr1/assemblyhelpersplatform/countnullnotnull