2013-11-20 14:36:59,357 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2013-11-20 14:36:59,382 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop
2013-11-20 14:36:59,383 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2013-11-20 14:36:59,419 INFO hadoop.HadoopPlatform (HadoopPlatform.java:setUp(112)) - not using cluster
2013-11-20 14:37:00,031 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:00,033 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:00,046 INFO property.AppProps (AppProps.java:getAppID(162)) - using app.id: E79DAAA195F44539BE7C0C45C2B07F14
2013-11-20 14:37:00,286 INFO util.Version (Version.java:printBanner(78)) - Concurrent, Inc - Cascading 2.2.0
2013-11-20 14:37:00,290 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting
2013-11-20 14:37:00,291 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:00,292 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:00,293 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false"]
2013-11-20 14:37:00,293 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] parallel execution is enabled: false
2013-11-20 14:37:00,294 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting jobs: 1
2013-11-20 14:37:00,294 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] allocating threads: 1
2013-11-20 14:37:00,296 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] starting step: (1/1) ...platform/multimerge+false
2013-11-20 14:37:00,343 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(52)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2013-11-20 14:37:00,410 WARN snappy.LoadSnappy (LoadSnappy.java:<clinit>(46)) - Snappy native library not loaded
2013-11-20 14:37:00,416 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:00,425 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:00,631 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] submitted hadoop job: job_local1962084816_0001
2013-11-20 14:37:00,663 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:00,666 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1962084816_0001_m_000000_0
2013-11-20 14:37:00,720 INFO util.ProcessTree (ProcessTree.java:isSetsidSupported(63)) - setsid exited with exit code 0
2013-11-20 14:37:00,729 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4fef3a22
2013-11-20 14:37:00,736 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:00,737 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@464693f9
2013-11-20 14:37:00,746 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:00,755 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:00,892 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:00,892 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:00,915 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:00,916 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:00,984 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:00,984 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2013-11-20 14:37:00,989 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:01,001 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:01,002 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 197, max: 672
2013-11-20 14:37:01,002 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,004 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:01,004 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 197, max: 672
2013-11-20 14:37:01,004 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,005 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:01,006 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 197, max: 672
2013-11-20 14:37:01,006 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,007 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:01,007 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 197, max: 672
2013-11-20 14:37:01,007 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,008 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:01,021 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:01,025 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1962084816_0001_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:01,032 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:01,032 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1962084816_0001_m_000000_0' done.
2013-11-20 14:37:01,033 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1962084816_0001_m_000000_0
2013-11-20 14:37:01,033 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1962084816_0001_m_000001_0
2013-11-20 14:37:01,036 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4cad4d40
2013-11-20 14:37:01,038 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:01,039 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4453d3cf
2013-11-20 14:37:01,041 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:01,041 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:01,171 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:01,171 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:01,173 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:01,173 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:01,199 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:01,199 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2013-11-20 14:37:01,200 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:01,318 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:01,318 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 75, total: 292, max: 672
2013-11-20 14:37:01,319 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,320 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:01,320 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 75, total: 292, max: 672
2013-11-20 14:37:01,321 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,322 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:01,322 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 75, total: 292, max: 672
2013-11-20 14:37:01,322 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,323 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:01,324 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 75, total: 292, max: 672
2013-11-20 14:37:01,324 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,325 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:01,329 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:01,331 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1962084816_0001_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:01,334 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:01,335 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1962084816_0001_m_000001_0' done.
2013-11-20 14:37:01,335 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1962084816_0001_m_000001_0
2013-11-20 14:37:01,335 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:01,350 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1fb88122
2013-11-20 14:37:01,351 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,356 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:01,362 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 472 bytes
2013-11-20 14:37:01,362 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,363 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:01,364 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:01,388 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2013-11-20 14:37:01,389 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false"]
2013-11-20 14:37:01,411 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1962084816_0001_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:01,414 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,414 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1962084816_0001_r_000000_0 is allowed to commit now
2013-11-20 14:37:01,418 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1962084816_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false
2013-11-20 14:37:01,419 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:01,419 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1962084816_0001_r_000000_0' done.
2013-11-20 14:37:01,444 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false/_temporary
2013-11-20 14:37:01,466 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:01,488 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:01,491 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multimerge+false
2013-11-20 14:37:01,518 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:01,519 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:01,548 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting
2013-11-20 14:37:01,549 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:01,549 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring"]
2013-11-20 14:37:01,549 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] parallel execution is enabled: false
2013-11-20 14:37:01,549 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting jobs: 1
2013-11-20 14:37:01,549 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] allocating threads: 1
2013-11-20 14:37:01,550 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2013-11-20 14:37:01,567 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:01,661 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] submitted hadoop job: job_local723932147_0002
2013-11-20 14:37:01,664 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:01,664 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local723932147_0002_m_000000_0
2013-11-20 14:37:01,666 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4d2ef4c6
2013-11-20 14:37:01,668 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:01,668 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5dd6c336
2013-11-20 14:37:01,671 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:01,671 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:01,813 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:01,813 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:01,814 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:01,815 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:01,827 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:01,827 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2013-11-20 14:37:01,827 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:01,829 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:01,829 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 387, max: 672
2013-11-20 14:37:01,829 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,830 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:01,830 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 84, total: 387, max: 672
2013-11-20 14:37:01,831 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:01,831 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:01,834 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:01,837 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local723932147_0002_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:01,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:01,840 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local723932147_0002_m_000000_0' done.
2013-11-20 14:37:01,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local723932147_0002_m_000000_0
2013-11-20 14:37:01,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:01,845 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5082277
2013-11-20 14:37:01,846 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,847 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:01,847 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2013-11-20 14:37:01,848 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,848 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:01,848 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:01,860 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2013-11-20 14:37:01,860 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring"]
2013-11-20 14:37:01,869 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local723932147_0002_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:01,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:01,871 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local723932147_0002_r_000000_0 is allowed to commit now
2013-11-20 14:37:01,874 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local723932147_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring
2013-11-20 14:37:01,876 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:01,876 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local723932147_0002_r_000000_0' done.
2013-11-20 14:37:01,886 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring/_temporary
2013-11-20 14:37:01,908 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:01,928 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:01,930 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/minbystring
2013-11-20 14:37:01,953 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:01,954 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:01,990 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] starting
2013-11-20 14:37:01,991 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:01,991 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:01,992 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage"]
2013-11-20 14:37:01,992 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] parallel execution is enabled: false
2013-11-20 14:37:01,993 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] starting jobs: 1
2013-11-20 14:37:01,993 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average-lhs+average-rhs] allocating threads: 1
2013-11-20 14:37:01,994 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2013-11-20 14:37:02,008 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,010 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,056 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average-lhs+average-rhs] submitted hadoop job: job_local1422836561_0003
2013-11-20 14:37:02,060 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:02,061 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1422836561_0003_m_000000_0
2013-11-20 14:37:02,062 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@12910b0b
2013-11-20 14:37:02,064 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:02,064 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2f8bbc98
2013-11-20 14:37:02,066 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:02,066 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:02,194 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:02,194 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:02,195 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:02,196 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:02,209 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:02,210 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2013-11-20 14:37:02,210 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:02,211 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:02,211 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 482, max: 672
2013-11-20 14:37:02,212 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,212 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:02,213 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 482, max: 672
2013-11-20 14:37:02,213 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,213 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:02,214 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 482, max: 672
2013-11-20 14:37:02,214 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,214 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:02,215 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 482, max: 672
2013-11-20 14:37:02,215 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,215 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:02,219 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:02,221 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1422836561_0003_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:02,224 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:02,224 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1422836561_0003_m_000000_0' done.
2013-11-20 14:37:02,224 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1422836561_0003_m_000000_0
2013-11-20 14:37:02,224 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1422836561_0003_m_000001_0
2013-11-20 14:37:02,226 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@675ee9e3
2013-11-20 14:37:02,227 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:02,227 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1ee99d0f
2013-11-20 14:37:02,229 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:02,229 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:02,345 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:02,346 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:02,347 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:02,347 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:02,362 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:02,362 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2013-11-20 14:37:02,363 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:02,367 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:02,367 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 44, total: 535, max: 672
2013-11-20 14:37:02,367 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,368 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:02,369 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 44, total: 535, max: 672
2013-11-20 14:37:02,369 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,369 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:02,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 44, total: 535, max: 672
2013-11-20 14:37:02,370 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:02,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 44, total: 535, max: 672
2013-11-20 14:37:02,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:02,372 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:02,374 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:02,376 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1422836561_0003_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:02,379 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:02,379 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1422836561_0003_m_000001_0' done.
2013-11-20 14:37:02,379 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1422836561_0003_m_000001_0
2013-11-20 14:37:02,379 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:02,387 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@205074de
2013-11-20 14:37:02,388 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:02,389 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:02,390 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 400 bytes
2013-11-20 14:37:02,390 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:02,391 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:02,391 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:02,403 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2013-11-20 14:37:02,404 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage"]
2013-11-20 14:37:02,412 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1422836561_0003_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:02,413 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:02,414 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1422836561_0003_r_000000_0 is allowed to commit now
2013-11-20 14:37:02,417 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1422836561_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage
2013-11-20 14:37:02,418 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:02,418 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1422836561_0003_r_000000_0' done.
2013-11-20 14:37:02,426 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage/_temporary
2013-11-20 14:37:02,438 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,451 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,453 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergeaverage
2013-11-20 14:37:02,466 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:02,466 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:02,486 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:02,486 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:02,487 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall"]
2013-11-20 14:37:02,487 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:02,487 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:02,487 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:02,488 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2013-11-20 14:37:02,504 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,543 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local1714398664_0004
2013-11-20 14:37:02,546 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:02,546 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1714398664_0004_m_000000_0
2013-11-20 14:37:02,549 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@30fd981a
2013-11-20 14:37:02,551 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:02,551 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@47959352
2013-11-20 14:37:02,552 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:02,557 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:02,557 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:02,564 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:02,565 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall"]
2013-11-20 14:37:02,568 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1714398664_0004_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:02,570 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:02,570 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1714398664_0004_m_000000_0 is allowed to commit now
2013-11-20 14:37:02,573 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1714398664_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall
2013-11-20 14:37:02,574 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:02,575 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1714398664_0004_m_000000_0' done.
2013-11-20 14:37:02,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1714398664_0004_m_000000_0
2013-11-20 14:37:02,575 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:02,589 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall/_temporary
2013-11-20 14:37:02,600 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,602 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/renameall
2013-11-20 14:37:02,617 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:02,618 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:02,682 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting
2013-11-20 14:37:02,683 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:02,684 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount"]
2013-11-20 14:37:02,685 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] parallel execution is enabled: false
2013-11-20 14:37:02,685 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting jobs: 2
2013-11-20 14:37:02,686 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] allocating threads: 1
2013-11-20 14:37:02,687 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] starting step: (1/2)
2013-11-20 14:37:02,701 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:02,740 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] submitted hadoop job: job_local1150837076_0005
2013-11-20 14:37:02,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:02,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1150837076_0005_m_000000_0
2013-11-20 14:37:02,745 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@ef1df6
2013-11-20 14:37:02,747 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:02,747 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7c6d75b6
2013-11-20 14:37:02,749 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:02,749 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:03,094 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:03,097 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:03,099 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:03,099 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,109 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:03,110 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2013-11-20 14:37:03,110 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:03,111 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:03,112 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 100, total: 204, max: 672
2013-11-20 14:37:03,112 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,113 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:03,113 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 100, total: 204, max: 672
2013-11-20 14:37:03,113 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,114 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:03,114 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 100, total: 204, max: 672
2013-11-20 14:37:03,114 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,114 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:03,115 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 100, total: 204, max: 672
2013-11-20 14:37:03,115 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,115 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:03,118 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:03,120 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1150837076_0005_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,122 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:03,123 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1150837076_0005_m_000000_0' done.
2013-11-20 14:37:03,123 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1150837076_0005_m_000000_0
2013-11-20 14:37:03,123 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:03,127 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7987b5ff
2013-11-20 14:37:03,127 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,129 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:03,129 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2013-11-20 14:37:03,130 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,130 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:03,130 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,140 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2013-11-20 14:37:03,140 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][5008039409/count/]
2013-11-20 14:37:03,150 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:03,152 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:03,152 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 99, total: 204, max: 672
2013-11-20 14:37:03,153 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 30% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,154 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1150837076_0005_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,156 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,156 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1150837076_0005_r_000000_0 is allowed to commit now
2013-11-20 14:37:03,158 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1150837076_0005_r_000000_0' to file:/tmp/hadoop-teamcity/5008039409_count_B245D76AE54F48C598315706AE1B8B31
2013-11-20 14:37:03,160 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:03,160 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1150837076_0005_r_000000_0' done.
2013-11-20 14:37:03,171 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] starting step: (2/2) ...elpersplatform/countcount
2013-11-20 14:37:03,237 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,270 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] submitted hadoop job: job_local2146166481_0006
2013-11-20 14:37:03,274 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:03,274 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2146166481_0006_m_000000_0
2013-11-20 14:37:03,278 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5b1d329d
2013-11-20 14:37:03,280 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5008039409_count_B245D76AE54F48C598315706AE1B8B31/part-00000
2013-11-20 14:37:03,281 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b8a80d4
2013-11-20 14:37:03,290 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:03,291 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:03,417 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:03,417 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:03,420 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:03,420 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,430 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][5008039409/count/]
2013-11-20 14:37:03,430 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2013-11-20 14:37:03,432 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:03,434 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:03,436 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2146166481_0006_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/5008039409_count_B245D76AE54F48C598315706AE1B8B31/part-00000:0+112
2013-11-20 14:37:03,438 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2146166481_0006_m_000000_0' done.
2013-11-20 14:37:03,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2146166481_0006_m_000000_0
2013-11-20 14:37:03,439 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:03,442 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@59c83354
2013-11-20 14:37:03,443 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,444 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:03,444 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 29 bytes
2013-11-20 14:37:03,444 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,445 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:03,445 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,454 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2013-11-20 14:37:03,454 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount"]
2013-11-20 14:37:03,460 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2146166481_0006_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,462 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,462 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2146166481_0006_r_000000_0 is allowed to commit now
2013-11-20 14:37:03,465 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2146166481_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount
2013-11-20 14:37:03,466 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:03,466 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2146166481_0006_r_000000_0' done.
2013-11-20 14:37:03,476 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount/_temporary
2013-11-20 14:37:03,488 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,500 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,502 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countcount
2013-11-20 14:37:03,516 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:03,517 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:03,545 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] starting
2013-11-20 14:37:03,546 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:03,546 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:03,546 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount"]
2013-11-20 14:37:03,546 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] parallel execution is enabled: false
2013-11-20 14:37:03,547 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] starting jobs: 1
2013-11-20 14:37:03,547 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs+count-rhs] allocating threads: 1
2013-11-20 14:37:03,548 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2013-11-20 14:37:03,563 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,565 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,596 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs+count-rhs] submitted hadoop job: job_local956412201_0007
2013-11-20 14:37:03,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:03,600 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local956412201_0007_m_000000_0
2013-11-20 14:37:03,601 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1f4b52aa
2013-11-20 14:37:03,603 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:03,603 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@478dabf1
2013-11-20 14:37:03,604 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:03,605 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:03,727 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:03,728 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:03,729 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:03,729 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:03,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:03,742 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:03,743 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:03,743 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 395, max: 672
2013-11-20 14:37:03,743 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:03,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 395, max: 672
2013-11-20 14:37:03,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:03,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 395, max: 672
2013-11-20 14:37:03,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:03,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 71, total: 395, max: 672
2013-11-20 14:37:03,746 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,747 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:03,750 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:03,752 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local956412201_0007_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,754 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:03,754 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local956412201_0007_m_000000_0' done.
2013-11-20 14:37:03,755 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local956412201_0007_m_000000_0
2013-11-20 14:37:03,755 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local956412201_0007_m_000001_0
2013-11-20 14:37:03,757 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6efde050
2013-11-20 14:37:03,758 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:03,758 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@188d92e
2013-11-20 14:37:03,760 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:03,760 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:03,880 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:03,880 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:03,882 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:03,882 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,895 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:03,895 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:03,895 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:03,899 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:03,899 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 65, total: 490, max: 672
2013-11-20 14:37:03,900 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,900 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:03,901 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 65, total: 490, max: 672
2013-11-20 14:37:03,901 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,902 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:03,902 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 65, total: 490, max: 672
2013-11-20 14:37:03,902 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,902 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:03,903 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 65, total: 490, max: 672
2013-11-20 14:37:03,903 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:03,903 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:03,906 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:03,908 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local956412201_0007_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:03,911 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:03,911 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local956412201_0007_m_000001_0' done.
2013-11-20 14:37:03,911 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local956412201_0007_m_000001_0
2013-11-20 14:37:03,912 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:03,919 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@26c94114
2013-11-20 14:37:03,920 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,921 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:03,922 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 238 bytes
2013-11-20 14:37:03,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,923 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:03,923 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:03,934 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:03,935 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount"]
2013-11-20 14:37:03,943 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local956412201_0007_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:03,944 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:03,945 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local956412201_0007_r_000000_0 is allowed to commit now
2013-11-20 14:37:03,948 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local956412201_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount
2013-11-20 14:37:03,949 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:03,950 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local956412201_0007_r_000000_0' done.
2013-11-20 14:37:03,966 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount/_temporary
2013-11-20 14:37:03,978 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,992 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:03,994 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergecount
2013-11-20 14:37:04,009 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:04,010 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:04,035 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting
2013-11-20 14:37:04,035 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:04,035 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring"]
2013-11-20 14:37:04,036 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] parallel execution is enabled: false
2013-11-20 14:37:04,036 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting jobs: 1
2013-11-20 14:37:04,036 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] allocating threads: 1
2013-11-20 14:37:04,037 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2013-11-20 14:37:04,050 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,087 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] submitted hadoop job: job_local1677683601_0008
2013-11-20 14:37:04,091 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:04,091 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1677683601_0008_m_000000_0
2013-11-20 14:37:04,093 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7d420fea
2013-11-20 14:37:04,095 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:04,095 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4b721a26
2013-11-20 14:37:04,096 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:04,097 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:04,480 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:04,480 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:04,481 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:04,482 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:04,491 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:04,491 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2013-11-20 14:37:04,491 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:04,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:04,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 155, total: 260, max: 672
2013-11-20 14:37:04,494 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:04,495 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:04,495 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 155, total: 260, max: 672
2013-11-20 14:37:04,495 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 38% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:04,496 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:04,498 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:04,500 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1677683601_0008_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:04,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:04,503 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1677683601_0008_m_000000_0' done.
2013-11-20 14:37:04,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1677683601_0008_m_000000_0
2013-11-20 14:37:04,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:04,510 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@561279c8
2013-11-20 14:37:04,511 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,512 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:04,512 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2013-11-20 14:37:04,513 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,513 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:04,513 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:04,521 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2013-11-20 14:37:04,522 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring"]
2013-11-20 14:37:04,528 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1677683601_0008_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:04,530 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,530 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1677683601_0008_r_000000_0 is allowed to commit now
2013-11-20 14:37:04,533 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1677683601_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring
2013-11-20 14:37:04,535 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:04,535 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1677683601_0008_r_000000_0' done.
2013-11-20 14:37:04,542 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring/_temporary
2013-11-20 14:37:04,551 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,562 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,564 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/maxbystring
2013-11-20 14:37:04,576 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:04,577 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:04,602 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] starting
2013-11-20 14:37:04,603 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:04,603 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] sink: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstn"]
2013-11-20 14:37:04,605 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] parallel execution is enabled: false
2013-11-20 14:37:04,605 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] starting jobs: 1
2013-11-20 14:37:04,605 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [first] allocating threads: 1
2013-11-20 14:37:04,606 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [first] starting step: (1/1) ...blyhelpersplatform/firstn
2013-11-20 14:37:04,618 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,645 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [first] submitted hadoop job: job_local70027179_0009
2013-11-20 14:37:04,649 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:04,649 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local70027179_0009_m_000000_0
2013-11-20 14:37:04,650 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@729e4f7c
2013-11-20 14:37:04,651 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:04,652 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4302a01f
2013-11-20 14:37:04,653 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:04,653 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:04,729 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:04,730 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:04,733 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:04,733 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:04,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:04,741 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2013-11-20 14:37:04,742 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:04,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:04,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 138, total: 355, max: 672
2013-11-20 14:37:04,744 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:04,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:04,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 138, total: 355, max: 672
2013-11-20 14:37:04,745 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 52% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:04,746 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:04,748 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:04,751 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local70027179_0009_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:04,753 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:04,753 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local70027179_0009_m_000000_0' done.
2013-11-20 14:37:04,753 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local70027179_0009_m_000000_0
2013-11-20 14:37:04,754 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:04,757 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@246bc73b
2013-11-20 14:37:04,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,759 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:04,759 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 122 bytes
2013-11-20 14:37:04,760 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,760 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:04,760 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:04,768 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2013-11-20 14:37:04,769 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstn"]
2013-11-20 14:37:04,778 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local70027179_0009_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:04,780 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,780 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local70027179_0009_r_000000_0 is allowed to commit now
2013-11-20 14:37:04,783 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local70027179_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstn
2013-11-20 14:37:04,785 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:04,785 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local70027179_0009_r_000000_0' done.
2013-11-20 14:37:04,799 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstn/_temporary
2013-11-20 14:37:04,810 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,812 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/firstn
2013-11-20 14:37:04,824 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:04,824 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:04,844 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] starting
2013-11-20 14:37:04,845 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:04,846 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields"]
2013-11-20 14:37:04,846 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] parallel execution is enabled: false
2013-11-20 14:37:04,846 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] starting jobs: 1
2013-11-20 14:37:04,846 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] allocating threads: 1
2013-11-20 14:37:04,847 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2013-11-20 14:37:04,860 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,887 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [coerce] submitted hadoop job: job_local1118403217_0010
2013-11-20 14:37:04,892 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:04,893 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1118403217_0010_m_000000_0
2013-11-20 14:37:04,894 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@435c41b
2013-11-20 14:37:04,896 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:04,897 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@49e61582
2013-11-20 14:37:04,898 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:04,904 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:04,904 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:04,914 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:04,914 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields"]
2013-11-20 14:37:04,923 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1118403217_0010_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:04,925 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:04,925 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1118403217_0010_m_000000_0 is allowed to commit now
2013-11-20 14:37:04,928 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1118403217_0010_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields
2013-11-20 14:37:04,930 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:04,930 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1118403217_0010_m_000000_0' done.
2013-11-20 14:37:04,930 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1118403217_0010_m_000000_0
2013-11-20 14:37:04,930 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:04,940 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields/_temporary
2013-11-20 14:37:04,951 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:04,953 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/coercefields
2013-11-20 14:37:04,964 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:04,964 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:04,979 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting
2013-11-20 14:37:04,980 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:04,980 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count"]
2013-11-20 14:37:04,981 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] parallel execution is enabled: false
2013-11-20 14:37:04,981 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting jobs: 1
2013-11-20 14:37:04,981 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] allocating threads: 1
2013-11-20 14:37:04,981 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2013-11-20 14:37:04,993 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,018 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] submitted hadoop job: job_local360229692_0011
2013-11-20 14:37:05,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:05,022 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local360229692_0011_m_000000_0
2013-11-20 14:37:05,024 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@61a8a9e7
2013-11-20 14:37:05,025 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:05,026 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@34a8a271
2013-11-20 14:37:05,027 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:05,028 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:05,149 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:05,150 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:05,151 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:05,151 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,160 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2013-11-20 14:37:05,161 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:05,162 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:05,162 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 113, total: 450, max: 672
2013-11-20 14:37:05,162 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:05,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:05,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 113, total: 450, max: 672
2013-11-20 14:37:05,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:05,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:05,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 113, total: 450, max: 672
2013-11-20 14:37:05,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:05,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:05,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 113, total: 450, max: 672
2013-11-20 14:37:05,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 66% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:05,166 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:05,168 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:05,170 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local360229692_0011_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,172 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:05,173 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local360229692_0011_m_000000_0' done.
2013-11-20 14:37:05,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local360229692_0011_m_000000_0
2013-11-20 14:37:05,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:05,177 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@45e4d960
2013-11-20 14:37:05,178 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,179 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:05,179 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2013-11-20 14:37:05,180 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,180 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:05,181 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,188 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2013-11-20 14:37:05,189 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count"]
2013-11-20 14:37:05,194 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local360229692_0011_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,196 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,196 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local360229692_0011_r_000000_0 is allowed to commit now
2013-11-20 14:37:05,199 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local360229692_0011_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count
2013-11-20 14:37:05,201 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:05,201 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local360229692_0011_r_000000_0' done.
2013-11-20 14:37:05,213 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count/_temporary
2013-11-20 14:37:05,222 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,232 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,233 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/count
2013-11-20 14:37:05,245 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:05,246 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:05,261 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting
2013-11-20 14:37:05,261 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,262 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby"]
2013-11-20 14:37:05,262 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] parallel execution is enabled: false
2013-11-20 14:37:05,262 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] starting jobs: 1
2013-11-20 14:37:05,262 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [max] allocating threads: 1
2013-11-20 14:37:05,263 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2013-11-20 14:37:05,274 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,302 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [max] submitted hadoop job: job_local1036881441_0012
2013-11-20 14:37:05,305 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:05,305 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1036881441_0012_m_000000_0
2013-11-20 14:37:05,307 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@29e07d3e
2013-11-20 14:37:05,309 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:05,309 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@22a7fdef
2013-11-20 14:37:05,310 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:05,310 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:05,434 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:05,434 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:05,435 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:05,436 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,444 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,445 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2013-11-20 14:37:05,445 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:05,446 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:05,446 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 96, total: 545, max: 672
2013-11-20 14:37:05,447 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:05,447 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 96, total: 545, max: 672
2013-11-20 14:37:05,448 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:05,448 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 96, total: 545, max: 672
2013-11-20 14:37:05,449 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:05,449 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 96, total: 545, max: 672
2013-11-20 14:37:05,449 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:05,452 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:05,453 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1036881441_0012_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,455 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:05,456 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1036881441_0012_m_000000_0' done.
2013-11-20 14:37:05,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1036881441_0012_m_000000_0
2013-11-20 14:37:05,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:05,460 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@40671416
2013-11-20 14:37:05,460 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,461 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:05,461 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 155 bytes
2013-11-20 14:37:05,462 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,462 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:05,462 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,472 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2013-11-20 14:37:05,473 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby"]
2013-11-20 14:37:05,478 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1036881441_0012_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,480 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,480 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1036881441_0012_r_000000_0 is allowed to commit now
2013-11-20 14:37:05,483 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1036881441_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby
2013-11-20 14:37:05,484 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:05,484 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1036881441_0012_r_000000_0' done.
2013-11-20 14:37:05,491 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby/_temporary
2013-11-20 14:37:05,501 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,513 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,515 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/maxby
2013-11-20 14:37:05,527 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:05,527 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:05,542 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] starting
2013-11-20 14:37:05,543 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,543 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby"]
2013-11-20 14:37:05,544 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] parallel execution is enabled: false
2013-11-20 14:37:05,544 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] starting jobs: 1
2013-11-20 14:37:05,544 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [min] allocating threads: 1
2013-11-20 14:37:05,545 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2013-11-20 14:37:05,557 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,587 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [min] submitted hadoop job: job_local950993387_0013
2013-11-20 14:37:05,590 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:05,590 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local950993387_0013_m_000000_0
2013-11-20 14:37:05,592 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@a0834e7
2013-11-20 14:37:05,594 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:05,594 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@73a80183
2013-11-20 14:37:05,595 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:05,596 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:05,719 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:05,719 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:05,721 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:05,721 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,728 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,728 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2013-11-20 14:37:05,729 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:05,730 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:05,730 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 47, total: 608, max: 672
2013-11-20 14:37:05,731 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:05,731 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 47, total: 608, max: 672
2013-11-20 14:37:05,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:05,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 47, total: 608, max: 672
2013-11-20 14:37:05,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:05,732 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 47, total: 608, max: 672
2013-11-20 14:37:05,733 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:05,735 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:05,737 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local950993387_0013_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:05,739 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local950993387_0013_m_000000_0' done.
2013-11-20 14:37:05,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local950993387_0013_m_000000_0
2013-11-20 14:37:05,739 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:05,743 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@d0da1d8
2013-11-20 14:37:05,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,744 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:05,745 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 155 bytes
2013-11-20 14:37:05,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,745 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:05,746 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:05,753 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2013-11-20 14:37:05,754 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby"]
2013-11-20 14:37:05,759 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local950993387_0013_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:05,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:05,761 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local950993387_0013_r_000000_0 is allowed to commit now
2013-11-20 14:37:05,764 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local950993387_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby
2013-11-20 14:37:05,765 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:05,766 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local950993387_0013_r_000000_0' done.
2013-11-20 14:37:05,771 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby/_temporary
2013-11-20 14:37:05,782 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,794 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,796 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/minby
2013-11-20 14:37:05,808 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:05,808 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:05,825 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] starting
2013-11-20 14:37:05,825 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:05,826 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum"]
2013-11-20 14:37:05,826 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] parallel execution is enabled: false
2013-11-20 14:37:05,826 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] starting jobs: 1
2013-11-20 14:37:05,826 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] allocating threads: 1
2013-11-20 14:37:05,827 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2013-11-20 14:37:05,837 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:05,867 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum] submitted hadoop job: job_local425925420_0014
2013-11-20 14:37:05,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:05,871 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local425925420_0014_m_000000_0
2013-11-20 14:37:05,872 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@39813c12
2013-11-20 14:37:05,874 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:05,874 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@67219b8c
2013-11-20 14:37:05,875 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:05,876 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:06,162 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:06,162 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:06,164 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,164 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,173 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:06,173 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2013-11-20 14:37:06,173 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:06,174 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:06,175 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 71, total: 175, max: 672
2013-11-20 14:37:06,175 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,176 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:06,176 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 71, total: 175, max: 672
2013-11-20 14:37:06,176 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,177 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:06,177 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 71, total: 175, max: 672
2013-11-20 14:37:06,177 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,177 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:06,178 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 71, total: 175, max: 672
2013-11-20 14:37:06,178 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,178 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:06,181 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:06,183 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local425925420_0014_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,185 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:06,185 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local425925420_0014_m_000000_0' done.
2013-11-20 14:37:06,185 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local425925420_0014_m_000000_0
2013-11-20 14:37:06,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:06,189 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@34786e16
2013-11-20 14:37:06,190 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,191 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:06,191 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2013-11-20 14:37:06,192 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,192 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:06,192 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,201 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2013-11-20 14:37:06,201 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum"]
2013-11-20 14:37:06,207 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local425925420_0014_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,209 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,209 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local425925420_0014_r_000000_0 is allowed to commit now
2013-11-20 14:37:06,212 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local425925420_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum
2013-11-20 14:37:06,214 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:06,214 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local425925420_0014_r_000000_0' done.
2013-11-20 14:37:06,229 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum/_temporary
2013-11-20 14:37:06,239 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,259 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,261 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/sum
2013-11-20 14:37:06,274 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:06,274 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:06,291 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] starting
2013-11-20 14:37:06,292 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:06,292 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:06,292 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum"]
2013-11-20 14:37:06,292 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] parallel execution is enabled: false
2013-11-20 14:37:06,293 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] starting jobs: 1
2013-11-20 14:37:06,293 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum-lhs+sum-rhs] allocating threads: 1
2013-11-20 14:37:06,293 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2013-11-20 14:37:06,302 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,303 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,327 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local357835142_0015
2013-11-20 14:37:06,330 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:06,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local357835142_0015_m_000000_0
2013-11-20 14:37:06,332 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@40e746a2
2013-11-20 14:37:06,334 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:06,334 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14d733ca
2013-11-20 14:37:06,335 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:06,336 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:06,375 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:06,375 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:06,377 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,377 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,386 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:06,386 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2013-11-20 14:37:06,386 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:06,388 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:06,388 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 100, total: 218, max: 672
2013-11-20 14:37:06,388 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:06,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 99, total: 218, max: 672
2013-11-20 14:37:06,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:06,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 99, total: 218, max: 672
2013-11-20 14:37:06,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:06,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 99, total: 218, max: 672
2013-11-20 14:37:06,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 32% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,392 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:06,394 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:06,396 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local357835142_0015_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:06,399 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local357835142_0015_m_000000_0' done.
2013-11-20 14:37:06,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local357835142_0015_m_000000_0
2013-11-20 14:37:06,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local357835142_0015_m_000001_0
2013-11-20 14:37:06,400 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@37ff6df2
2013-11-20 14:37:06,402 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:06,402 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5b1cd8cc
2013-11-20 14:37:06,403 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:06,403 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:06,478 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:06,478 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:06,479 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,479 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,489 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:06,489 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2013-11-20 14:37:06,490 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:06,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:06,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 313, max: 672
2013-11-20 14:37:06,494 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,494 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:06,495 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 313, max: 672
2013-11-20 14:37:06,495 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,496 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:06,496 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 313, max: 672
2013-11-20 14:37:06,496 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,496 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:06,497 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 313, max: 672
2013-11-20 14:37:06,497 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,497 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:06,500 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:06,501 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local357835142_0015_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:06,503 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:06,503 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local357835142_0015_m_000001_0' done.
2013-11-20 14:37:06,504 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local357835142_0015_m_000001_0
2013-11-20 14:37:06,504 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:06,510 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3a780024
2013-11-20 14:37:06,511 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,512 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:06,513 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 238 bytes
2013-11-20 14:37:06,513 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,513 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:06,513 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,523 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2013-11-20 14:37:06,523 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum"]
2013-11-20 14:37:06,530 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local357835142_0015_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,532 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,532 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local357835142_0015_r_000000_0 is allowed to commit now
2013-11-20 14:37:06,535 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local357835142_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum
2013-11-20 14:37:06,536 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:06,537 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local357835142_0015_r_000000_0' done.
2013-11-20 14:37:06,546 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum/_temporary
2013-11-20 14:37:06,556 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,567 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,569 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergesum
2013-11-20 14:37:06,581 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:06,581 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:06,594 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow"]
2013-11-20 14:37:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:06,595 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:06,596 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2013-11-20 14:37:06,604 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,631 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local1218795706_0016
2013-11-20 14:37:06,633 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:06,633 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1218795706_0016_m_000000_0
2013-11-20 14:37:06,635 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@592b12d
2013-11-20 14:37:06,636 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:06,636 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4ecfe790
2013-11-20 14:37:06,637 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:06,640 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,640 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,645 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:06,645 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow"]
2013-11-20 14:37:06,647 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1218795706_0016_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,648 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,649 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1218795706_0016_m_000000_0 is allowed to commit now
2013-11-20 14:37:06,651 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1218795706_0016_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow
2013-11-20 14:37:06,654 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:06,655 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1218795706_0016_m_000000_0' done.
2013-11-20 14:37:06,655 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1218795706_0016_m_000000_0
2013-11-20 14:37:06,655 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:06,663 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow/_temporary
2013-11-20 14:37:06,673 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,675 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/discardnarrow
2013-11-20 14:37:06,686 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:06,687 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:06,697 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:06,697 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:06,698 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename"]
2013-11-20 14:37:06,698 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:06,698 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:06,698 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:06,701 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2013-11-20 14:37:06,710 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,737 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local1643580439_0017
2013-11-20 14:37:06,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:06,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1643580439_0017_m_000000_0
2013-11-20 14:37:06,742 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@440f52ed
2013-11-20 14:37:06,743 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:06,744 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1cc21a68
2013-11-20 14:37:06,745 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:06,748 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,748 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,753 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:06,754 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename"]
2013-11-20 14:37:06,755 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1643580439_0017_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,757 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:06,757 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1643580439_0017_m_000000_0 is allowed to commit now
2013-11-20 14:37:06,759 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1643580439_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename
2013-11-20 14:37:06,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:06,761 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1643580439_0017_m_000000_0' done.
2013-11-20 14:37:06,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1643580439_0017_m_000000_0
2013-11-20 14:37:06,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:06,770 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename/_temporary
2013-11-20 14:37:06,780 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,781 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/rename
2013-11-20 14:37:06,796 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:06,797 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:06,821 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting
2013-11-20 14:37:06,822 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:06,822 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:06,822 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true"]
2013-11-20 14:37:06,822 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] parallel execution is enabled: false
2013-11-20 14:37:06,822 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting jobs: 1
2013-11-20 14:37:06,823 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] allocating threads: 1
2013-11-20 14:37:06,823 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] starting step: (1/1) ...splatform/multimerge+true
2013-11-20 14:37:06,837 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,838 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:06,867 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] submitted hadoop job: job_local517661455_0018
2013-11-20 14:37:06,870 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:06,877 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local517661455_0018_m_000000_0
2013-11-20 14:37:06,882 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@221e5f0
2013-11-20 14:37:06,884 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:06,884 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2c618b08
2013-11-20 14:37:06,885 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:06,885 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:06,919 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:06,920 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:06,922 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:06,922 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:06,933 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:06,933 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2013-11-20 14:37:06,934 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:06,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:06,936 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 107, total: 314, max: 672
2013-11-20 14:37:06,936 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,937 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:06,937 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 107, total: 314, max: 672
2013-11-20 14:37:06,938 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,938 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:06,939 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 107, total: 314, max: 672
2013-11-20 14:37:06,939 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,940 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:06,940 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 107, total: 314, max: 672
2013-11-20 14:37:06,940 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:06,941 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:06,943 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:06,945 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local517661455_0018_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:06,947 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:06,947 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local517661455_0018_m_000000_0' done.
2013-11-20 14:37:06,947 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local517661455_0018_m_000000_0
2013-11-20 14:37:06,948 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local517661455_0018_m_000001_0
2013-11-20 14:37:06,949 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2f46ccac
2013-11-20 14:37:06,950 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:06,950 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@63654886
2013-11-20 14:37:06,951 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:06,952 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:07,078 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:07,079 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:07,081 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:07,081 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,091 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:07,091 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2013-11-20 14:37:07,092 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:07,096 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:07,096 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 101, total: 409, max: 672
2013-11-20 14:37:07,096 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 60% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,098 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:07,098 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 101, total: 409, max: 672
2013-11-20 14:37:07,098 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 60% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,099 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:07,099 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 101, total: 409, max: 672
2013-11-20 14:37:07,100 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 60% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,101 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:07,101 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 101, total: 409, max: 672
2013-11-20 14:37:07,101 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 60% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,102 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:07,104 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:07,106 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local517661455_0018_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:07,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:07,109 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local517661455_0018_m_000001_0' done.
2013-11-20 14:37:07,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local517661455_0018_m_000001_0
2013-11-20 14:37:07,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:07,116 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@75f9eba0
2013-11-20 14:37:07,117 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,118 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:07,120 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 472 bytes
2013-11-20 14:37:07,120 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,121 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:07,121 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,133 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2013-11-20 14:37:07,133 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true"]
2013-11-20 14:37:07,142 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local517661455_0018_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,144 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,144 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local517661455_0018_r_000000_0 is allowed to commit now
2013-11-20 14:37:07,147 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local517661455_0018_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true
2013-11-20 14:37:07,148 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:07,148 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local517661455_0018_r_000000_0' done.
2013-11-20 14:37:07,154 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true/_temporary
2013-11-20 14:37:07,163 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,172 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,174 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multimerge+true
2013-11-20 14:37:07,186 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:07,187 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:07,216 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] starting
2013-11-20 14:37:07,216 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,216 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount"]
2013-11-20 14:37:07,216 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] parallel execution is enabled: false
2013-11-20 14:37:07,217 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] starting jobs: 2
2013-11-20 14:37:07,217 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [second] allocating threads: 1
2013-11-20 14:37:07,217 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [second] starting step: (1/2)
2013-11-20 14:37:07,225 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,247 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [second] submitted hadoop job: job_local1559029346_0019
2013-11-20 14:37:07,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:07,249 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1559029346_0019_m_000000_0
2013-11-20 14:37:07,250 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1f071537
2013-11-20 14:37:07,252 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:07,252 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@dd1e765
2013-11-20 14:37:07,253 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:07,254 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:07,375 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:07,375 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:07,377 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:07,377 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,384 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,385 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2013-11-20 14:37:07,385 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:07,388 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:07,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 504, max: 672
2013-11-20 14:37:07,389 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 75% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:07,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 504, max: 672
2013-11-20 14:37:07,390 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 75% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:07,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 504, max: 672
2013-11-20 14:37:07,391 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 75% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,392 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:07,392 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 504, max: 672
2013-11-20 14:37:07,392 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 75% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2013-11-20 14:37:07,393 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:07,395 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:07,396 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1559029346_0019_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:07,399 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1559029346_0019_m_000000_0' done.
2013-11-20 14:37:07,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1559029346_0019_m_000000_0
2013-11-20 14:37:07,399 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:07,402 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@66bccad2
2013-11-20 14:37:07,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,403 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:07,404 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2013-11-20 14:37:07,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,404 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:07,405 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,411 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2013-11-20 14:37:07,411 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5950528351/first/]
2013-11-20 14:37:07,415 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1559029346_0019_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,417 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,417 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1559029346_0019_r_000000_0 is allowed to commit now
2013-11-20 14:37:07,418 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1559029346_0019_r_000000_0' to file:/tmp/hadoop-teamcity/5950528351_first_BCF357BF7237422E8E4CD8549DFD3887
2013-11-20 14:37:07,419 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:07,419 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1559029346_0019_r_000000_0' done.
2013-11-20 14:37:07,431 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [second] starting step: (2/2) ...form/samesourcemergecount
2013-11-20 14:37:07,438 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,462 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [second] submitted hadoop job: job_local2094355393_0020
2013-11-20 14:37:07,464 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:07,464 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2094355393_0020_m_000000_0
2013-11-20 14:37:07,466 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@a8e13ab
2013-11-20 14:37:07,467 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5950528351_first_BCF357BF7237422E8E4CD8549DFD3887/part-00000
2013-11-20 14:37:07,467 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@218fcdce
2013-11-20 14:37:07,468 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:07,468 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:07,589 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:07,590 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:07,591 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:07,591 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,600 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5950528351/first/]
2013-11-20 14:37:07,601 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2013-11-20 14:37:07,601 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:07,602 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:07,602 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 69, total: 599, max: 672
2013-11-20 14:37:07,603 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:07,603 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 69, total: 599, max: 672
2013-11-20 14:37:07,603 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:07,605 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:07,607 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2094355393_0020_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,609 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/5950528351_first_BCF357BF7237422E8E4CD8549DFD3887/part-00000:0+160
2013-11-20 14:37:07,609 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2094355393_0020_m_000000_0' done.
2013-11-20 14:37:07,609 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2094355393_0020_m_000000_0
2013-11-20 14:37:07,609 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:07,612 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7476a6d9
2013-11-20 14:37:07,613 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,613 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:07,614 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2013-11-20 14:37:07,614 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,615 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:07,615 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,623 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2013-11-20 14:37:07,623 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount"]
2013-11-20 14:37:07,628 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2094355393_0020_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,629 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,629 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2094355393_0020_r_000000_0 is allowed to commit now
2013-11-20 14:37:07,631 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2094355393_0020_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount
2013-11-20 14:37:07,633 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:07,633 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2094355393_0020_r_000000_0' done.
2013-11-20 14:37:07,647 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount/_temporary
2013-11-20 14:37:07,657 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,667 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,669 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/samesourcemergecount
2013-11-20 14:37:07,678 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:07,679 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:07,688 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] starting
2013-11-20 14:37:07,689 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,689 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average"]
2013-11-20 14:37:07,689 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] parallel execution is enabled: false
2013-11-20 14:37:07,689 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] starting jobs: 1
2013-11-20 14:37:07,689 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] allocating threads: 1
2013-11-20 14:37:07,690 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2013-11-20 14:37:07,697 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,730 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average] submitted hadoop job: job_local388210068_0021
2013-11-20 14:37:07,732 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:07,732 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local388210068_0021_m_000000_0
2013-11-20 14:37:07,733 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@33a626ac
2013-11-20 14:37:07,734 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:07,735 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@192277ed
2013-11-20 14:37:07,736 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:07,736 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:07,772 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:07,773 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:07,775 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:07,775 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,783 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,783 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2013-11-20 14:37:07,784 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:07,785 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:07,785 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 150, total: 646, max: 672
2013-11-20 14:37:07,786 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:07,786 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 150, total: 646, max: 672
2013-11-20 14:37:07,787 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:07,787 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 150, total: 646, max: 672
2013-11-20 14:37:07,788 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:07,788 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 150, total: 646, max: 672
2013-11-20 14:37:07,788 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:07,790 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:07,792 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local388210068_0021_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,794 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:07,794 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local388210068_0021_m_000000_0' done.
2013-11-20 14:37:07,794 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local388210068_0021_m_000000_0
2013-11-20 14:37:07,794 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:07,797 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4f47af3
2013-11-20 14:37:07,798 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,798 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:07,799 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 200 bytes
2013-11-20 14:37:07,799 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,799 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:07,799 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,805 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2013-11-20 14:37:07,805 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average"]
2013-11-20 14:37:07,810 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local388210068_0021_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,811 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local388210068_0021_r_000000_0 is allowed to commit now
2013-11-20 14:37:07,813 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local388210068_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average
2013-11-20 14:37:07,814 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:07,814 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local388210068_0021_r_000000_0' done.
2013-11-20 14:37:07,823 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average/_temporary
2013-11-20 14:37:07,832 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,842 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,844 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/average
2013-11-20 14:37:07,855 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:07,856 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:07,867 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] starting
2013-11-20 14:37:07,867 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,868 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls"]
2013-11-20 14:37:07,868 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] parallel execution is enabled: false
2013-11-20 14:37:07,868 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] starting jobs: 1
2013-11-20 14:37:07,868 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [sum] allocating threads: 1
2013-11-20 14:37:07,869 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2013-11-20 14:37:07,876 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:07,898 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [sum] submitted hadoop job: job_local1188760715_0022
2013-11-20 14:37:07,900 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:07,901 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1188760715_0022_m_000000_0
2013-11-20 14:37:07,902 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@77158bbe
2013-11-20 14:37:07,903 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:07,903 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@56a57bb2
2013-11-20 14:37:07,904 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:07,904 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:07,944 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:07,945 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:07,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:07,946 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,952 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:07,952 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2013-11-20 14:37:07,953 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:07,959 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:07,959 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 159, total: 652, max: 672
2013-11-20 14:37:07,960 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:07,960 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 159, total: 652, max: 672
2013-11-20 14:37:07,961 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:07,961 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 159, total: 652, max: 672
2013-11-20 14:37:07,961 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:07,962 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 159, total: 652, max: 672
2013-11-20 14:37:07,962 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:07,964 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:07,965 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1188760715_0022_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:07,967 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1188760715_0022_m_000000_0' done.
2013-11-20 14:37:07,967 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1188760715_0022_m_000000_0
2013-11-20 14:37:07,968 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:07,971 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5511e6b9
2013-11-20 14:37:07,972 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,973 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:07,973 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 116 bytes
2013-11-20 14:37:07,973 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,974 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:07,974 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:07,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2013-11-20 14:37:07,983 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls"]
2013-11-20 14:37:07,988 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1188760715_0022_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:07,990 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:07,990 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1188760715_0022_r_000000_0 is allowed to commit now
2013-11-20 14:37:07,993 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1188760715_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls
2013-11-20 14:37:07,994 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:07,994 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1188760715_0022_r_000000_0' done.
2013-11-20 14:37:08,005 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls/_temporary
2013-11-20 14:37:08,013 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,023 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,025 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/sumnulls
2013-11-20 14:37:08,035 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,035 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,044 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting
2013-11-20 14:37:08,045 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,045 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall"]
2013-11-20 14:37:08,045 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] parallel execution is enabled: false
2013-11-20 14:37:08,045 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting jobs: 1
2013-11-20 14:37:08,045 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] allocating threads: 1
2013-11-20 14:37:08,046 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2013-11-20 14:37:08,054 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,078 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] submitted hadoop job: job_local484100519_0023
2013-11-20 14:37:08,080 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,081 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local484100519_0023_m_000000_0
2013-11-20 14:37:08,082 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1c0693a5
2013-11-20 14:37:08,083 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:08,083 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@14654dec
2013-11-20 14:37:08,084 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:08,084 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:08,124 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:08,124 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:08,125 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,125 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,131 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,131 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2013-11-20 14:37:08,132 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:08,133 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:08,135 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:08,136 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local484100519_0023_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:08,138 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local484100519_0023_m_000000_0' done.
2013-11-20 14:37:08,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local484100519_0023_m_000000_0
2013-11-20 14:37:08,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,141 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@487011c
2013-11-20 14:37:08,142 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,142 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:08,143 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 8 bytes
2013-11-20 14:37:08,143 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,144 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:08,144 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,150 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2013-11-20 14:37:08,150 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall"]
2013-11-20 14:37:08,153 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local484100519_0023_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,154 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,155 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local484100519_0023_r_000000_0 is allowed to commit now
2013-11-20 14:37:08,157 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local484100519_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall
2013-11-20 14:37:08,157 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:08,158 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local484100519_0023_r_000000_0' done.
2013-11-20 14:37:08,164 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall/_temporary
2013-11-20 14:37:08,174 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,247 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,248 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countall
2013-11-20 14:37:08,259 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,260 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,273 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting
2013-11-20 14:37:08,273 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,273 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi"]
2013-11-20 14:37:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] parallel execution is enabled: false
2013-11-20 14:37:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] starting jobs: 1
2013-11-20 14:37:08,274 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [name] allocating threads: 1
2013-11-20 14:37:08,274 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2013-11-20 14:37:08,282 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,305 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [name] submitted hadoop job: job_local621884822_0024
2013-11-20 14:37:08,308 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,308 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local621884822_0024_m_000000_0
2013-11-20 14:37:08,309 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@523bd44c
2013-11-20 14:37:08,310 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:08,310 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@59a2a780
2013-11-20 14:37:08,312 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:08,312 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:08,402 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:08,402 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:08,404 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,405 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,413 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,413 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2013-11-20 14:37:08,414 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:08,415 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:08,415 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 694, max: 694
2013-11-20 14:37:08,417 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:08,417 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 694, max: 694
2013-11-20 14:37:08,418 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:08,418 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 694, max: 694
2013-11-20 14:37:08,418 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:08,419 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 694, max: 694
2013-11-20 14:37:08,420 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:08,422 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:08,423 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local621884822_0024_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,425 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:08,425 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local621884822_0024_m_000000_0' done.
2013-11-20 14:37:08,425 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local621884822_0024_m_000000_0
2013-11-20 14:37:08,425 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,428 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@556be8a0
2013-11-20 14:37:08,429 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,430 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:08,430 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 452 bytes
2013-11-20 14:37:08,430 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,431 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:08,431 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,439 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2013-11-20 14:37:08,439 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi"]
2013-11-20 14:37:08,445 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local621884822_0024_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,447 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,447 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local621884822_0024_r_000000_0 is allowed to commit now
2013-11-20 14:37:08,449 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local621884822_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi
2013-11-20 14:37:08,450 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:08,450 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local621884822_0024_r_000000_0' done.
2013-11-20 14:37:08,460 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi/_temporary
2013-11-20 14:37:08,470 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,481 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,483 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multi
2013-11-20 14:37:08,492 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,493 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,503 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:08,503 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,503 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow"]
2013-11-20 14:37:08,504 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:08,504 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:08,504 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:08,504 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2013-11-20 14:37:08,513 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,539 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local2060820010_0025
2013-11-20 14:37:08,544 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,545 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2060820010_0025_m_000000_0
2013-11-20 14:37:08,546 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6569a85d
2013-11-20 14:37:08,547 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:08,547 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@73c9c556
2013-11-20 14:37:08,549 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:08,551 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,551 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,556 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,556 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow"]
2013-11-20 14:37:08,558 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2060820010_0025_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,559 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,559 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2060820010_0025_m_000000_0 is allowed to commit now
2013-11-20 14:37:08,562 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2060820010_0025_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow
2013-11-20 14:37:08,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:08,564 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2060820010_0025_m_000000_0' done.
2013-11-20 14:37:08,564 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2060820010_0025_m_000000_0
2013-11-20 14:37:08,565 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,575 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow/_temporary
2013-11-20 14:37:08,584 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,585 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/retainnarrow
2013-11-20 14:37:08,595 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,595 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,604 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:08,604 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,604 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow"]
2013-11-20 14:37:08,604 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:08,605 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:08,605 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2013-11-20 14:37:08,613 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,638 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local852727530_0026
2013-11-20 14:37:08,640 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,641 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local852727530_0026_m_000000_0
2013-11-20 14:37:08,642 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@775a9fdf
2013-11-20 14:37:08,643 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:08,643 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@799318fa
2013-11-20 14:37:08,644 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:08,647 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,647 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,651 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,651 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow"]
2013-11-20 14:37:08,653 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local852727530_0026_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,654 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,654 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local852727530_0026_m_000000_0 is allowed to commit now
2013-11-20 14:37:08,656 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local852727530_0026_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow
2013-11-20 14:37:08,657 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:08,658 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local852727530_0026_m_000000_0' done.
2013-11-20 14:37:08,658 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local852727530_0026_m_000000_0
2013-11-20 14:37:08,658 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,671 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow/_temporary
2013-11-20 14:37:08,680 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/renamenarrow
2013-11-20 14:37:08,693 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,694 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,705 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] starting
2013-11-20 14:37:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull"]
2013-11-20 14:37:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] parallel execution is enabled: false
2013-11-20 14:37:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] starting jobs: 1
2013-11-20 14:37:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [average] allocating threads: 1
2013-11-20 14:37:08,708 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2013-11-20 14:37:08,716 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,742 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [average] submitted hadoop job: job_local2017614821_0027
2013-11-20 14:37:08,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2017614821_0027_m_000000_0
2013-11-20 14:37:08,746 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@72d37f4b
2013-11-20 14:37:08,747 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:08,747 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3b5c8049
2013-11-20 14:37:08,748 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:08,749 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:08,784 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:08,785 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:08,786 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,786 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,794 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:08,794 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2013-11-20 14:37:08,794 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:08,798 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:08,798 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 197, total: 694, max: 694
2013-11-20 14:37:08,799 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:08,799 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 197, total: 694, max: 694
2013-11-20 14:37:08,800 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:08,800 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 197, total: 694, max: 694
2013-11-20 14:37:08,801 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:08,801 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 197, total: 694, max: 694
2013-11-20 14:37:08,801 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:08,804 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:08,805 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2017614821_0027_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,807 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:08,807 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2017614821_0027_m_000000_0' done.
2013-11-20 14:37:08,807 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2017614821_0027_m_000000_0
2013-11-20 14:37:08,807 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,810 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3327d4d9
2013-11-20 14:37:08,811 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,812 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:08,813 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 200 bytes
2013-11-20 14:37:08,813 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,814 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:08,814 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2013-11-20 14:37:08,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull"]
2013-11-20 14:37:08,826 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2017614821_0027_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,828 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,828 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2017614821_0027_r_000000_0 is allowed to commit now
2013-11-20 14:37:08,830 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2017614821_0027_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull
2013-11-20 14:37:08,832 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:08,833 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2017614821_0027_r_000000_0' done.
2013-11-20 14:37:08,846 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull/_temporary
2013-11-20 14:37:08,854 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,864 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,865 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/averagenull
2013-11-20 14:37:08,874 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,875 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:08,883 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] starting
2013-11-20 14:37:08,883 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,883 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce"]
2013-11-20 14:37:08,884 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] parallel execution is enabled: false
2013-11-20 14:37:08,884 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] starting jobs: 1
2013-11-20 14:37:08,884 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [coerce] allocating threads: 1
2013-11-20 14:37:08,884 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2013-11-20 14:37:08,892 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,916 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [coerce] submitted hadoop job: job_local382265611_0028
2013-11-20 14:37:08,919 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:08,919 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local382265611_0028_m_000000_0
2013-11-20 14:37:08,921 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2ae61f87
2013-11-20 14:37:08,922 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt
2013-11-20 14:37:08,922 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@214f223f
2013-11-20 14:37:08,923 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2013-11-20 14:37:08,925 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:08,925 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:08,930 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt"]
2013-11-20 14:37:08,930 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce"]
2013-11-20 14:37:08,932 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local382265611_0028_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:08,933 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:08,933 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local382265611_0028_m_000000_0 is allowed to commit now
2013-11-20 14:37:08,935 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local382265611_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce
2013-11-20 14:37:08,936 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2013-11-20 14:37:08,937 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local382265611_0028_m_000000_0' done.
2013-11-20 14:37:08,937 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local382265611_0028_m_000000_0
2013-11-20 14:37:08,937 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:08,949 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce/_temporary
2013-11-20 14:37:08,957 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:08,959 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/coerce
2013-11-20 14:37:08,971 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:08,972 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:09,021 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] starting
2013-11-20 14:37:09,021 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:09,022 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway"]
2013-11-20 14:37:09,022 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] parallel execution is enabled: false
2013-11-20 14:37:09,022 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] starting jobs: 5
2013-11-20 14:37:09,022 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count-lhs*source*count...] allocating threads: 1
2013-11-20 14:37:09,023 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] starting step: (1/5)
2013-11-20 14:37:09,030 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,049 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] submitted hadoop job: job_local1130262406_0029
2013-11-20 14:37:09,052 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,052 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1130262406_0029_m_000000_0
2013-11-20 14:37:09,054 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6f683e80
2013-11-20 14:37:09,055 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:09,055 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4977e3d4
2013-11-20 14:37:09,056 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,056 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,093 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,093 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,094 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,094 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,100 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:09,100 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2013-11-20 14:37:09,100 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:09,101 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:09,101 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 70, total: 694, max: 694
2013-11-20 14:37:09,102 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:09,102 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 70, total: 694, max: 694
2013-11-20 14:37:09,103 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2013-11-20 14:37:09,103 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 70, total: 694, max: 694
2013-11-20 14:37:09,103 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2013-11-20 14:37:09,104 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 70, total: 694, max: 694
2013-11-20 14:37:09,104 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,106 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,108 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1130262406_0029_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,109 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:09,109 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1130262406_0029_m_000000_0' done.
2013-11-20 14:37:09,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1130262406_0029_m_000000_0
2013-11-20 14:37:09,110 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,113 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@63871f70
2013-11-20 14:37:09,113 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,114 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:09,114 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2013-11-20 14:37:09,115 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,115 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,115 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,123 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2013-11-20 14:37:09,124 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][8981516325/source/]
2013-11-20 14:37:09,128 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1130262406_0029_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,130 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,130 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1130262406_0029_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,132 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1130262406_0029_r_000000_0' to file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A
2013-11-20 14:37:09,133 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,133 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1130262406_0029_r_000000_0' done.
2013-11-20 14:37:09,143 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] starting step: (3/5)
2013-11-20 14:37:09,151 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,170 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] submitted hadoop job: job_local1362682189_0030
2013-11-20 14:37:09,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,173 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1362682189_0030_m_000000_0
2013-11-20 14:37:09,174 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@449b8dc1
2013-11-20 14:37:09,176 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000
2013-11-20 14:37:09,176 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5c8ab562
2013-11-20 14:37:09,177 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,179 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,219 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,219 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,220 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,221 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,226 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][8981516325/source/]
2013-11-20 14:37:09,226 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2013-11-20 14:37:09,226 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:09,227 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:09,227 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 201, total: 693, max: 693
2013-11-20 14:37:09,227 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:09,228 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 201, total: 693, max: 693
2013-11-20 14:37:09,228 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,230 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,231 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1362682189_0030_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,232 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000:0+160
2013-11-20 14:37:09,233 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1362682189_0030_m_000000_0' done.
2013-11-20 14:37:09,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1362682189_0030_m_000000_0
2013-11-20 14:37:09,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,236 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6d4a4067
2013-11-20 14:37:09,236 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,237 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:09,237 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2013-11-20 14:37:09,238 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,238 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,238 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,243 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2013-11-20 14:37:09,243 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][3900686091/count-lhs/]
2013-11-20 14:37:09,247 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1362682189_0030_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,248 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,248 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1362682189_0030_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,250 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1362682189_0030_r_000000_0' to file:/tmp/hadoop-teamcity/3900686091_count_lhs_98F0FDFA90D24EC4BA2E4C02CA056178
2013-11-20 14:37:09,251 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,251 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1362682189_0030_r_000000_0' done.
2013-11-20 14:37:09,256 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] starting step: (4/5)
2013-11-20 14:37:09,263 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,282 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] submitted hadoop job: job_local205416782_0031
2013-11-20 14:37:09,284 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local205416782_0031_m_000000_0
2013-11-20 14:37:09,286 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4cd522dd
2013-11-20 14:37:09,287 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000
2013-11-20 14:37:09,287 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5b8c0b02
2013-11-20 14:37:09,288 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,288 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,324 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,324 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,325 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,325 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,330 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][8981516325/source/]
2013-11-20 14:37:09,331 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2013-11-20 14:37:09,331 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:09,332 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2013-11-20 14:37:09,332 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 693, max: 693
2013-11-20 14:37:09,332 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2013-11-20 14:37:09,332 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 93, total: 693, max: 693
2013-11-20 14:37:09,333 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,335 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,337 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local205416782_0031_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,338 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000:0+160
2013-11-20 14:37:09,338 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local205416782_0031_m_000000_0' done.
2013-11-20 14:37:09,339 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local205416782_0031_m_000000_0
2013-11-20 14:37:09,339 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,342 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@19af724
2013-11-20 14:37:09,342 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,343 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:09,343 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2013-11-20 14:37:09,344 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,344 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,344 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,349 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2013-11-20 14:37:09,349 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][3132488567/count-rhs/]
2013-11-20 14:37:09,353 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local205416782_0031_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,354 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,354 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local205416782_0031_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,355 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local205416782_0031_r_000000_0' to file:/tmp/hadoop-teamcity/3132488567_count_rhs_C530E5E80EC1456E8148763BA3F9B2C6
2013-11-20 14:37:09,356 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,357 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local205416782_0031_r_000000_0' done.
2013-11-20 14:37:09,365 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] starting step: (2/5)
2013-11-20 14:37:09,373 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,374 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,393 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] submitted hadoop job: job_local1277212593_0032
2013-11-20 14:37:09,396 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,396 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1277212593_0032_m_000000_0
2013-11-20 14:37:09,397 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4ead4391
2013-11-20 14:37:09,398 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000
2013-11-20 14:37:09,399 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3675463
2013-11-20 14:37:09,400 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,400 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,444 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,444 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,447 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,447 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,454 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][8981516325/source/]
2013-11-20 14:37:09,454 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:09,456 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,459 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,461 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1277212593_0032_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,463 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/8981516325_source_C5C43C48C26C4D3F81BAA63E3D55BC4A/part-00000:0+160
2013-11-20 14:37:09,464 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1277212593_0032_m_000000_0' done.
2013-11-20 14:37:09,465 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1277212593_0032_m_000000_0
2013-11-20 14:37:09,466 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1277212593_0032_m_000001_0
2013-11-20 14:37:09,467 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7796649
2013-11-20 14:37:09,469 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/3132488567_count_rhs_C530E5E80EC1456E8148763BA3F9B2C6/part-00000
2013-11-20 14:37:09,470 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@396bfded
2013-11-20 14:37:09,471 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,472 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,507 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,508 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,509 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,510 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,514 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][3132488567/count-rhs/]
2013-11-20 14:37:09,515 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:09,516 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,518 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,519 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1277212593_0032_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:09,521 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/3132488567_count_rhs_C530E5E80EC1456E8148763BA3F9B2C6/part-00000:0+160
2013-11-20 14:37:09,521 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1277212593_0032_m_000001_0' done.
2013-11-20 14:37:09,521 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1277212593_0032_m_000001_0
2013-11-20 14:37:09,522 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,527 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4086d2c7
2013-11-20 14:37:09,527 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,528 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:09,529 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 154 bytes
2013-11-20 14:37:09,529 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,530 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,530 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,535 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2013-11-20 14:37:09,535 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][8200801617/source_count-rhs/]
2013-11-20 14:37:09,544 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2013-11-20 14:37:09,545 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2013-11-20 14:37:09,551 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1277212593_0032_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,552 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,552 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1277212593_0032_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,554 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1277212593_0032_r_000000_0' to file:/tmp/hadoop-teamcity/8200801617_source_count_rhs_D897F50E06FF4870A8937B378FF479EC
2013-11-20 14:37:09,558 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,558 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1277212593_0032_r_000000_0' done.
2013-11-20 14:37:09,562 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2013-11-20 14:37:09,569 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,570 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,589 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count-lhs*source*count...] submitted hadoop job: job_local87620624_0033
2013-11-20 14:37:09,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local87620624_0033_m_000000_0
2013-11-20 14:37:09,594 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4a3e7799
2013-11-20 14:37:09,595 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/8200801617_source_count_rhs_D897F50E06FF4870A8937B378FF479EC/part-00000
2013-11-20 14:37:09,595 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3a86edfe
2013-11-20 14:37:09,597 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,597 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,638 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,639 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,640 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,640 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,646 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][8200801617/source_count-rhs/]
2013-11-20 14:37:09,647 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2013-11-20 14:37:09,648 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,650 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,651 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local87620624_0033_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,652 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/8200801617_source_count_rhs_D897F50E06FF4870A8937B378FF479EC/part-00000:0+200
2013-11-20 14:37:09,653 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local87620624_0033_m_000000_0' done.
2013-11-20 14:37:09,653 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local87620624_0033_m_000000_0
2013-11-20 14:37:09,653 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local87620624_0033_m_000001_0
2013-11-20 14:37:09,654 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3986c40d
2013-11-20 14:37:09,655 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/3900686091_count_lhs_98F0FDFA90D24EC4BA2E4C02CA056178/part-00000
2013-11-20 14:37:09,656 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@35497537
2013-11-20 14:37:09,657 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,657 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,693 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,694 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,695 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,695 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,701 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][3900686091/count-lhs/]
2013-11-20 14:37:09,701 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2013-11-20 14:37:09,703 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,704 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,706 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local87620624_0033_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:09,707 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/3900686091_count_lhs_98F0FDFA90D24EC4BA2E4C02CA056178/part-00000:0+160
2013-11-20 14:37:09,708 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local87620624_0033_m_000001_0' done.
2013-11-20 14:37:09,708 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local87620624_0033_m_000001_0
2013-11-20 14:37:09,708 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,713 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@72e8cc94
2013-11-20 14:37:09,713 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,714 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:09,715 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 194 bytes
2013-11-20 14:37:09,715 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,716 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,716 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,721 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2013-11-20 14:37:09,721 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway"]
2013-11-20 14:37:09,724 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2013-11-20 14:37:09,724 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2013-11-20 14:37:09,726 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local87620624_0033_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,727 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,727 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local87620624_0033_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,729 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local87620624_0033_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway
2013-11-20 14:37:09,730 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,730 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local87620624_0033_r_000000_0' done.
2013-11-20 14:37:09,738 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2013-11-20 14:37:09,746 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,755 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,756 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/samesourcemergethreeway
2013-11-20 14:37:09,767 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:09,768 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:09,781 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] starting
2013-11-20 14:37:09,782 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:09,782 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:09,782 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2013-11-20 14:37:09,783 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] parallel execution is enabled: false
2013-11-20 14:37:09,783 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] starting jobs: 1
2013-11-20 14:37:09,783 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [lhs+rhs] allocating threads: 1
2013-11-20 14:37:09,784 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2013-11-20 14:37:09,791 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,792 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,811 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [lhs+rhs] submitted hadoop job: job_local1472609673_0034
2013-11-20 14:37:09,814 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:09,814 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1472609673_0034_m_000000_0
2013-11-20 14:37:09,815 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5e0a14a
2013-11-20 14:37:09,816 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:09,816 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@61bb121f
2013-11-20 14:37:09,817 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,818 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,859 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,859 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,861 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,861 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,865 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:09,865 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2013-11-20 14:37:09,868 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,870 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,871 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1472609673_0034_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,873 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:09,873 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1472609673_0034_m_000000_0' done.
2013-11-20 14:37:09,873 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1472609673_0034_m_000000_0
2013-11-20 14:37:09,873 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1472609673_0034_m_000001_0
2013-11-20 14:37:09,874 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5ec1c697
2013-11-20 14:37:09,875 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt
2013-11-20 14:37:09,875 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@73a51852
2013-11-20 14:37:09,876 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:09,876 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:09,910 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:09,910 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:09,911 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:09,911 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,915 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt"]
2013-11-20 14:37:09,915 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2013-11-20 14:37:09,917 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:09,919 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:09,920 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1472609673_0034_m_000001_0 is done. And is in the process of commiting
2013-11-20 14:37:09,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2013-11-20 14:37:09,922 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1472609673_0034_m_000001_0' done.
2013-11-20 14:37:09,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1472609673_0034_m_000001_0
2013-11-20 14:37:09,922 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:09,927 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7b5a5d5d
2013-11-20 14:37:09,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,928 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2013-11-20 14:37:09,929 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2013-11-20 14:37:09,929 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,929 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:09,930 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:09,934 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2013-11-20 14:37:09,934 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2013-11-20 14:37:09,938 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1472609673_0034_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:09,939 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:09,940 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1472609673_0034_r_000000_0 is allowed to commit now
2013-11-20 14:37:09,941 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1472609673_0034_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic
2013-11-20 14:37:09,942 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:09,942 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1472609673_0034_r_000000_0' done.
2013-11-20 14:37:09,948 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2013-11-20 14:37:09,956 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:09,957 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic
2013-11-20 14:37:09,966 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:09,967 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:09,975 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting
2013-11-20 14:37:09,975 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:09,976 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique"]
2013-11-20 14:37:09,976 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] parallel execution is enabled: false
2013-11-20 14:37:09,976 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] starting jobs: 1
2013-11-20 14:37:09,977 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [shape] allocating threads: 1
2013-11-20 14:37:09,978 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2013-11-20 14:37:09,986 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:10,005 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [shape] submitted hadoop job: job_local29266498_0035
2013-11-20 14:37:10,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:10,007 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local29266498_0035_m_000000_0
2013-11-20 14:37:10,008 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4e5fe91a
2013-11-20 14:37:10,009 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:10,009 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@c115a1e
2013-11-20 14:37:10,010 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:10,011 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:10,053 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:10,053 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:10,054 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:10,054 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:10,058 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:10,058 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2013-11-20 14:37:10,060 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:10,062 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:10,063 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local29266498_0035_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:10,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:10,065 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local29266498_0035_m_000000_0' done.
2013-11-20 14:37:10,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local29266498_0035_m_000000_0
2013-11-20 14:37:10,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:10,068 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3d092ac9
2013-11-20 14:37:10,069 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,069 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:10,070 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2013-11-20 14:37:10,070 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,070 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:10,071 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:10,074 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2013-11-20 14:37:10,074 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique"]
2013-11-20 14:37:10,078 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local29266498_0035_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:10,079 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,080 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local29266498_0035_r_000000_0 is allowed to commit now
2013-11-20 14:37:10,081 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local29266498_0035_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique
2013-11-20 14:37:10,082 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:10,082 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local29266498_0035_r_000000_0' done.
2013-11-20 14:37:10,089 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique/_temporary
2013-11-20 14:37:10,097 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:10,099 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/unique
2013-11-20 14:37:10,109 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2013-11-20 14:37:10,110 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(203)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/artifacts-24/filestore/cascading/cascading-hadoop/2.2.0/jar/c7b52bd4c6da523ebde0a5d6eca2475123783db2/cascading-hadoop-2.2.0.jar
2013-11-20 14:37:10,119 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting
2013-11-20 14:37:10,119 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:10,120 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull"]
2013-11-20 14:37:10,121 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] parallel execution is enabled: false
2013-11-20 14:37:10,121 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] starting jobs: 1
2013-11-20 14:37:10,121 INFO flow.Flow (BaseFlow.java:logInfo(1306)) - [count] allocating threads: 1
2013-11-20 14:37:10,122 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] starting step: (1/1) ...platform/countnullnotnull
2013-11-20 14:37:10,129 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:10,148 INFO flow.FlowStep (BaseFlowStep.java:logInfo(742)) - [count] submitted hadoop job: job_local557237084_0036
2013-11-20 14:37:10,150 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2013-11-20 14:37:10,151 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local557237084_0036_m_000000_0
2013-11-20 14:37:10,152 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5c9c62da
2013-11-20 14:37:10,153 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt
2013-11-20 14:37:10,153 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4a8487ba
2013-11-20 14:37:10,154 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2013-11-20 14:37:10,154 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2013-11-20 14:37:10,189 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2013-11-20 14:37:10,190 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2013-11-20 14:37:10,191 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.2.0
2013-11-20 14:37:10,191 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2013-11-20 14:37:10,197 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt"]
2013-11-20 14:37:10,197 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2013-11-20 14:37:10,198 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2013-11-20 14:37:10,202 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2013-11-20 14:37:10,204 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2013-11-20 14:37:10,205 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local557237084_0036_m_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:10,207 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2013-11-20 14:37:10,207 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local557237084_0036_m_000000_0' done.
2013-11-20 14:37:10,207 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local557237084_0036_m_000000_0
2013-11-20 14:37:10,207 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2013-11-20 14:37:10,210 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@62af7285
2013-11-20 14:37:10,211 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,212 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2013-11-20 14:37:10,212 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 10 bytes
2013-11-20 14:37:10,213 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,213 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.2.0
2013-11-20 14:37:10,214 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2013-11-20 14:37:10,220 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2013-11-20 14:37:10,220 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull"]
2013-11-20 14:37:10,224 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local557237084_0036_r_000000_0 is done. And is in the process of commiting
2013-11-20 14:37:10,225 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2013-11-20 14:37:10,225 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local557237084_0036_r_000000_0 is allowed to commit now
2013-11-20 14:37:10,227 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local557237084_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull
2013-11-20 14:37:10,228 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2013-11-20 14:37:10,228 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local557237084_0036_r_000000_0' done.
2013-11-20 14:37:10,241 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull/_temporary
2013-11-20 14:37:10,250 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:10,259 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2013-11-20 14:37:10,261 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/d1f96f51023738a5/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countnullnotnull