2014-08-22 18:04:16,990 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2014-08-22 18:04:17,038 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop
2014-08-22 18:04:17,039 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.pipe.assembly.AssemblyHelpersPlatformTest
2014-08-22 18:04:17,079 INFO hadoop.HadoopPlatform (HadoopPlatform.java:setUp(112)) - not using cluster
2014-08-22 18:04:17,759 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:17,761 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:17,774 INFO property.AppProps (AppProps.java:getAppID(162)) - using app.id: 27888337D1D0481FA9F08816CB2B8FD1
2014-08-22 18:04:18,050 INFO util.Version (Version.java:printBanner(78)) - Concurrent, Inc - Cascading 2.5.6
2014-08-22 18:04:18,053 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-08-22 18:04:18,054 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:18,054 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:18,055 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false"]
2014-08-22 18:04:18,055 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] parallel execution is enabled: false
2014-08-22 18:04:18,055 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting jobs: 1
2014-08-22 18:04:18,055 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] allocating threads: 1
2014-08-22 18:04:18,057 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...platform/multimerge+false
2014-08-22 18:04:18,105 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(52)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-08-22 18:04:18,174 WARN snappy.LoadSnappy (LoadSnappy.java:<clinit>(46)) - Snappy native library not loaded
2014-08-22 18:04:18,178 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:18,188 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:18,414 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local840290949_0001
2014-08-22 18:04:18,415 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-08-22 18:04:18,449 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:18,453 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local840290949_0001_m_000000_0
2014-08-22 18:04:18,512 INFO util.ProcessTree (ProcessTree.java:isSetsidSupported(63)) - setsid exited with exit code 0
2014-08-22 18:04:18,520 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@815a3a1
2014-08-22 18:04:18,528 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:18,529 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1e9af0b1
2014-08-22 18:04:18,536 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:18,545 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:18,690 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:18,690 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:18,716 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:18,716 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:18,791 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:18,791 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-08-22 18:04:18,797 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:18,810 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:18,811 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 82, total: 197, max: 672
2014-08-22 18:04:18,811 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:18,813 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:18,813 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 82, total: 197, max: 672
2014-08-22 18:04:18,814 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:18,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:18,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 82, total: 197, max: 672
2014-08-22 18:04:18,815 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:18,816 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:18,816 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 82, total: 197, max: 672
2014-08-22 18:04:18,817 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 29% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:18,818 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:18,830 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:18,833 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local840290949_0001_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:18,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:18,840 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local840290949_0001_m_000000_0' done.
2014-08-22 18:04:18,840 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local840290949_0001_m_000000_0
2014-08-22 18:04:18,841 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local840290949_0001_m_000001_0
2014-08-22 18:04:18,843 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7059c8b5
2014-08-22 18:04:18,845 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:18,845 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@53e6978d
2014-08-22 18:04:18,847 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:18,848 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:18,977 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:18,978 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:18,980 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:18,980 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:19,009 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:19,009 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-08-22 18:04:19,010 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:19,136 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:19,137 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 73, total: 292, max: 672
2014-08-22 18:04:19,137 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,138 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:19,139 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 73, total: 292, max: 672
2014-08-22 18:04:19,139 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:19,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 73, total: 292, max: 672
2014-08-22 18:04:19,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,141 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:19,141 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 73, total: 292, max: 672
2014-08-22 18:04:19,142 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,143 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:19,147 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:19,149 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local840290949_0001_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:19,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:19,153 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local840290949_0001_m_000001_0' done.
2014-08-22 18:04:19,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local840290949_0001_m_000001_0
2014-08-22 18:04:19,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:19,177 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@139d115
2014-08-22 18:04:19,178 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,184 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:19,192 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 472 bytes
2014-08-22 18:04:19,193 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,194 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:19,195 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:19,223 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by: multi-lhs:[{1}:'char'] multi-rhs:[{1}:'char']]
2014-08-22 18:04:19,224 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false"]
2014-08-22 18:04:19,249 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local840290949_0001_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:19,252 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,253 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local840290949_0001_r_000000_0 is allowed to commit now
2014-08-22 18:04:19,258 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local840290949_0001_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false
2014-08-22 18:04:19,260 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:19,260 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local840290949_0001_r_000000_0' done.
2014-08-22 18:04:19,279 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+false/_temporary
2014-08-22 18:04:19,304 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,328 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,331 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multimerge+false
2014-08-22 18:04:19,361 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:19,361 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:19,395 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-08-22 18:04:19,396 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:19,397 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] sink: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring"]
2014-08-22 18:04:19,397 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] parallel execution is enabled: false
2014-08-22 18:04:19,397 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting jobs: 1
2014-08-22 18:04:19,398 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] allocating threads: 1
2014-08-22 18:04:19,400 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/minbystring
2014-08-22 18:04:19,416 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,518 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local33117270_0002
2014-08-22 18:04:19,519 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-08-22 18:04:19,521 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:19,522 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local33117270_0002_m_000000_0
2014-08-22 18:04:19,524 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@65cb048e
2014-08-22 18:04:19,525 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:19,526 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@221a5770
2014-08-22 18:04:19,528 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:19,528 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:19,663 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:19,664 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:19,665 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:19,665 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:19,679 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:19,679 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2014-08-22 18:04:19,680 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:19,682 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:19,682 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 387, max: 672
2014-08-22 18:04:19,682 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,683 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:19,683 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 81, total: 387, max: 672
2014-08-22 18:04:19,683 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:19,684 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:19,688 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:19,690 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local33117270_0002_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:19,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:19,693 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local33117270_0002_m_000000_0' done.
2014-08-22 18:04:19,693 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local33117270_0002_m_000000_0
2014-08-22 18:04:19,694 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:19,698 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@30db95a1
2014-08-22 18:04:19,699 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,700 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:19,701 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2014-08-22 18:04:19,701 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,702 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:19,702 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:19,717 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2014-08-22 18:04:19,718 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'min']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring"]
2014-08-22 18:04:19,764 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local33117270_0002_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:19,767 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:19,767 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local33117270_0002_r_000000_0 is allowed to commit now
2014-08-22 18:04:19,771 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local33117270_0002_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring
2014-08-22 18:04:19,773 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:19,773 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local33117270_0002_r_000000_0' done.
2014-08-22 18:04:19,784 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minbystring/_temporary
2014-08-22 18:04:19,807 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,829 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,831 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/minbystring
2014-08-22 18:04:19,858 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:19,858 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:19,900 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] starting
2014-08-22 18:04:19,900 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:19,900 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:19,901 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage"]
2014-08-22 18:04:19,901 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] parallel execution is enabled: false
2014-08-22 18:04:19,901 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] starting jobs: 1
2014-08-22 18:04:19,901 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average-lhs+average-rhs] allocating threads: 1
2014-08-22 18:04:19,902 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] starting step: (1/1) ...persplatform/mergeaverage
2014-08-22 18:04:19,918 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,920 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:19,978 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] submitted hadoop job: job_local1089358622_0003
2014-08-22 18:04:19,978 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average-lhs+average-rhs] tracking url: http://localhost:8080/
2014-08-22 18:04:19,981 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:19,981 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1089358622_0003_m_000000_0
2014-08-22 18:04:19,983 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5e1645b9
2014-08-22 18:04:19,985 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:19,985 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6bb5eba4
2014-08-22 18:04:19,986 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:19,987 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:20,118 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:20,118 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:20,120 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:20,120 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:20,136 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:20,136 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-08-22 18:04:20,137 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:20,138 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:20,138 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 91, total: 482, max: 672
2014-08-22 18:04:20,139 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:20,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 91, total: 482, max: 672
2014-08-22 18:04:20,140 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,141 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:20,141 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 91, total: 482, max: 672
2014-08-22 18:04:20,141 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,142 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:20,142 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 91, total: 482, max: 672
2014-08-22 18:04:20,142 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,143 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:20,147 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:20,150 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1089358622_0003_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:20,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:20,153 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1089358622_0003_m_000000_0' done.
2014-08-22 18:04:20,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1089358622_0003_m_000000_0
2014-08-22 18:04:20,153 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1089358622_0003_m_000001_0
2014-08-22 18:04:20,155 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4e62974e
2014-08-22 18:04:20,156 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:20,157 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@56dc64a2
2014-08-22 18:04:20,158 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:20,158 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:20,289 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:20,290 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:20,291 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:20,291 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:20,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:20,308 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-08-22 18:04:20,309 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:20,313 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:20,313 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 535, max: 672
2014-08-22 18:04:20,313 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,314 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:20,315 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 535, max: 672
2014-08-22 18:04:20,315 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,316 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:20,316 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 535, max: 672
2014-08-22 18:04:20,316 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,317 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:20,317 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 535, max: 672
2014-08-22 18:04:20,317 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 79% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:20,318 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:20,321 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:20,324 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1089358622_0003_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:20,326 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:20,327 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1089358622_0003_m_000001_0' done.
2014-08-22 18:04:20,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1089358622_0003_m_000001_0
2014-08-22 18:04:20,327 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:20,336 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@17332b63
2014-08-22 18:04:20,337 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:20,338 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:20,339 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 400 bytes
2014-08-22 18:04:20,339 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:20,340 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:20,340 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:20,354 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average-lhs+average-rhs)[by: average-lhs:[{1}:'char'] average-rhs:[{1}:'char']]
2014-08-22 18:04:20,354 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage"]
2014-08-22 18:04:20,363 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1089358622_0003_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:20,365 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:20,365 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1089358622_0003_r_000000_0 is allowed to commit now
2014-08-22 18:04:20,368 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1089358622_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage
2014-08-22 18:04:20,371 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:20,371 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1089358622_0003_r_000000_0' done.
2014-08-22 18:04:20,389 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergeaverage/_temporary
2014-08-22 18:04:20,403 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:20,418 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:20,420 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergeaverage
2014-08-22 18:04:20,435 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:20,436 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:20,460 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:20,460 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:20,461 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall"]
2014-08-22 18:04:20,461 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:20,461 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:20,462 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:20,464 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...helpersplatform/renameall
2014-08-22 18:04:20,478 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:20,522 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local237898580_0004
2014-08-22 18:04:20,524 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:20,527 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:20,527 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local237898580_0004_m_000000_0
2014-08-22 18:04:20,530 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6b064b80
2014-08-22 18:04:20,531 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:20,532 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@55a517bd
2014-08-22 18:04:20,533 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:20,539 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:20,540 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:20,549 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:20,550 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall"]
2014-08-22 18:04:20,553 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local237898580_0004_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:20,556 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:20,556 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local237898580_0004_m_000000_0 is allowed to commit now
2014-08-22 18:04:20,560 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local237898580_0004_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall
2014-08-22 18:04:20,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:20,562 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local237898580_0004_m_000000_0' done.
2014-08-22 18:04:20,562 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local237898580_0004_m_000000_0
2014-08-22 18:04:20,563 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:20,580 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renameall/_temporary
2014-08-22 18:04:20,612 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:20,614 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/renameall
2014-08-22 18:04:20,631 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:20,632 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:20,683 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-08-22 18:04:20,683 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:20,684 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount"]
2014-08-22 18:04:20,684 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-08-22 18:04:20,684 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 2
2014-08-22 18:04:20,685 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-08-22 18:04:20,685 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/2)
2014-08-22 18:04:20,701 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:20,739 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1363322548_0005
2014-08-22 18:04:20,739 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-08-22 18:04:20,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:20,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1363322548_0005_m_000000_0
2014-08-22 18:04:20,745 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@527a9c0f
2014-08-22 18:04:20,746 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:20,746 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3717ee94
2014-08-22 18:04:20,748 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:20,748 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:21,101 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:21,102 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:21,105 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:21,105 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,118 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:21,118 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2014-08-22 18:04:21,118 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:21,120 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:21,120 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 74, total: 178, max: 672
2014-08-22 18:04:21,120 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,121 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:21,121 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 74, total: 178, max: 672
2014-08-22 18:04:21,121 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,122 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:21,122 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 74, total: 178, max: 672
2014-08-22 18:04:21,122 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,123 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:21,123 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 74, total: 178, max: 672
2014-08-22 18:04:21,123 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,124 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:21,127 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:21,129 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1363322548_0005_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,132 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:21,132 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1363322548_0005_m_000000_0' done.
2014-08-22 18:04:21,132 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1363322548_0005_m_000000_0
2014-08-22 18:04:21,133 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:21,137 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@60c0a150
2014-08-22 18:04:21,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,139 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:21,139 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2014-08-22 18:04:21,140 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,140 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:21,141 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,152 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2014-08-22 18:04:21,152 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['count', 'count2']]"][3633115836/count/]
2014-08-22 18:04:21,163 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:21,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:21,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 178, max: 672
2014-08-22 18:04:21,165 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 26% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,167 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1363322548_0005_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,169 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,169 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1363322548_0005_r_000000_0 is allowed to commit now
2014-08-22 18:04:21,172 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1363322548_0005_r_000000_0' to file:/tmp/hadoop-teamcity/3633115836_count_B4F5E7F019A8444888BCE2CFF663D3C1
2014-08-22 18:04:21,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:21,174 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1363322548_0005_r_000000_0' done.
2014-08-22 18:04:21,186 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (2/2) ...elpersplatform/countcount
2014-08-22 18:04:21,200 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:21,235 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1636489852_0006
2014-08-22 18:04:21,236 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-08-22 18:04:21,240 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:21,240 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1636489852_0006_m_000000_0
2014-08-22 18:04:21,242 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@67bf6ec8
2014-08-22 18:04:21,244 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/3633115836_count_B4F5E7F019A8444888BCE2CFF663D3C1/part-00000
2014-08-22 18:04:21,244 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@214c4ac9
2014-08-22 18:04:21,251 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:21,251 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:21,381 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:21,381 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:21,383 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:21,383 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,394 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['count', 'count2']]"][3633115836/count/]
2014-08-22 18:04:21,394 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'count']]
2014-08-22 18:04:21,396 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:21,399 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:21,401 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1636489852_0006_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/3633115836_count_B4F5E7F019A8444888BCE2CFF663D3C1/part-00000:0+112
2014-08-22 18:04:21,404 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1636489852_0006_m_000000_0' done.
2014-08-22 18:04:21,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1636489852_0006_m_000000_0
2014-08-22 18:04:21,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:21,408 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@c481958
2014-08-22 18:04:21,409 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,410 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:21,410 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 29 bytes
2014-08-22 18:04:21,411 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,411 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:21,411 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,422 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'count']]
2014-08-22 18:04:21,422 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count', 'count2']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount"]
2014-08-22 18:04:21,429 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1636489852_0006_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,431 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,431 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1636489852_0006_r_000000_0 is allowed to commit now
2014-08-22 18:04:21,434 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1636489852_0006_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount
2014-08-22 18:04:21,436 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:21,436 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1636489852_0006_r_000000_0' done.
2014-08-22 18:04:21,452 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countcount/_temporary
2014-08-22 18:04:21,463 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:21,475 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:21,477 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countcount
2014-08-22 18:04:21,492 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:21,493 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:21,531 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] starting
2014-08-22 18:04:21,531 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:21,531 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:21,532 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount"]
2014-08-22 18:04:21,532 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] parallel execution is enabled: false
2014-08-22 18:04:21,532 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] starting jobs: 1
2014-08-22 18:04:21,532 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs+count-rhs] allocating threads: 1
2014-08-22 18:04:21,533 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] starting step: (1/1) ...elpersplatform/mergecount
2014-08-22 18:04:21,547 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:21,548 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:21,593 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] submitted hadoop job: job_local1199145718_0007
2014-08-22 18:04:21,593 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs+count-rhs] tracking url: http://localhost:8080/
2014-08-22 18:04:21,596 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:21,596 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1199145718_0007_m_000000_0
2014-08-22 18:04:21,598 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1e05b891
2014-08-22 18:04:21,599 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:21,600 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@99c873d
2014-08-22 18:04:21,601 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:21,602 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:21,741 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:21,741 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:21,743 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:21,743 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,758 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:21,758 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:21,759 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:21,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:21,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 392, max: 672
2014-08-22 18:04:21,760 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,761 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:21,762 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 392, max: 672
2014-08-22 18:04:21,762 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:21,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 392, max: 672
2014-08-22 18:04:21,763 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,764 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:21,764 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 89, total: 392, max: 672
2014-08-22 18:04:21,764 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 58% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,765 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:21,769 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:21,771 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1199145718_0007_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,774 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:21,774 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1199145718_0007_m_000000_0' done.
2014-08-22 18:04:21,774 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1199145718_0007_m_000000_0
2014-08-22 18:04:21,774 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1199145718_0007_m_000001_0
2014-08-22 18:04:21,776 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5dfa490
2014-08-22 18:04:21,779 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:21,779 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@316becfe
2014-08-22 18:04:21,781 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:21,782 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:21,913 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:21,914 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:21,915 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:21,915 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,929 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:21,929 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:21,930 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:21,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:21,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 83, total: 487, max: 672
2014-08-22 18:04:21,934 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:21,935 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 83, total: 487, max: 672
2014-08-22 18:04:21,936 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,936 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:21,937 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 83, total: 487, max: 672
2014-08-22 18:04:21,937 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,937 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:21,938 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 83, total: 487, max: 672
2014-08-22 18:04:21,938 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 72% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:21,938 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:21,941 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:21,943 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1199145718_0007_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:21,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:21,946 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1199145718_0007_m_000001_0' done.
2014-08-22 18:04:21,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1199145718_0007_m_000001_0
2014-08-22 18:04:21,946 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:21,954 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5603f377
2014-08-22 18:04:21,955 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,956 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:21,957 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 238 bytes
2014-08-22 18:04:21,958 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,958 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:21,959 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:21,973 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs+count-rhs)[by: count-lhs:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:21,973 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount"]
2014-08-22 18:04:21,981 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1199145718_0007_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:21,983 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:21,983 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1199145718_0007_r_000000_0 is allowed to commit now
2014-08-22 18:04:21,987 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1199145718_0007_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount
2014-08-22 18:04:21,989 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:21,989 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1199145718_0007_r_000000_0' done.
2014-08-22 18:04:22,001 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergecount/_temporary
2014-08-22 18:04:22,012 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,024 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,026 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergecount
2014-08-22 18:04:22,040 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:22,041 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:22,066 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-08-22 18:04:22,066 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:22,066 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] sink: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring"]
2014-08-22 18:04:22,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] parallel execution is enabled: false
2014-08-22 18:04:22,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting jobs: 1
2014-08-22 18:04:22,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] allocating threads: 1
2014-08-22 18:04:22,070 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...lpersplatform/maxbystring
2014-08-22 18:04:22,084 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,115 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local1137385539_0008
2014-08-22 18:04:22,116 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-08-22 18:04:22,123 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:22,123 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1137385539_0008_m_000000_0
2014-08-22 18:04:22,125 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@61f873dd
2014-08-22 18:04:22,126 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:22,126 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@15664f1a
2014-08-22 18:04:22,128 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:22,128 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:22,261 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:22,261 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:22,263 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:22,263 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:22,273 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:22,273 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'num']]
2014-08-22 18:04:22,274 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:22,275 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:22,275 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 558, max: 672
2014-08-22 18:04:22,276 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:22,277 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 42, total: 558, max: 672
2014-08-22 18:04:22,277 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:22,280 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:22,282 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1137385539_0008_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:22,284 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:22,285 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1137385539_0008_m_000000_0' done.
2014-08-22 18:04:22,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1137385539_0008_m_000000_0
2014-08-22 18:04:22,285 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:22,289 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3244331c
2014-08-22 18:04:22,290 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,291 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:22,291 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2014-08-22 18:04:22,292 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,292 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:22,292 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:22,301 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'num']]
2014-08-22 18:04:22,301 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'max']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring"]
2014-08-22 18:04:22,309 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1137385539_0008_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:22,311 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,311 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1137385539_0008_r_000000_0 is allowed to commit now
2014-08-22 18:04:22,314 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1137385539_0008_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring
2014-08-22 18:04:22,316 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:22,317 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1137385539_0008_r_000000_0' done.
2014-08-22 18:04:22,324 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxbystring/_temporary
2014-08-22 18:04:22,335 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,349 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,351 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/maxbystring
2014-08-22 18:04:22,367 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:22,368 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:22,399 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] starting
2014-08-22 18:04:22,400 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] source: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs+rhs-cross.txt"]
2014-08-22 18:04:22,400 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] sink: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstnfields"]
2014-08-22 18:04:22,401 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] parallel execution is enabled: false
2014-08-22 18:04:22,401 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] starting jobs: 1
2014-08-22 18:04:22,401 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [first] allocating threads: 1
2014-08-22 18:04:22,402 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] starting step: (1/1) ...persplatform/firstnfields
2014-08-22 18:04:22,414 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,450 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] submitted hadoop job: job_local2008798054_0009
2014-08-22 18:04:22,451 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [first] tracking url: http://localhost:8080/
2014-08-22 18:04:22,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:22,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2008798054_0009_m_000000_0
2014-08-22 18:04:22,459 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1b382d35
2014-08-22 18:04:22,461 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs+rhs-cross.txt
2014-08-22 18:04:22,461 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b963273
2014-08-22 18:04:22,462 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:22,463 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:22,806 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:22,807 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:22,810 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:22,810 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:22,820 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs+rhs-cross.txt"]
2014-08-22 18:04:22,820 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'num']]
2014-08-22 18:04:22,821 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:22,824 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:22,824 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 130, total: 234, max: 672
2014-08-22 18:04:22,825 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 34% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:22,826 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:22,827 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 130, total: 234, max: 672
2014-08-22 18:04:22,827 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 34% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:22,828 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:22,831 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:22,833 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2008798054_0009_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:22,835 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs+rhs-cross.txt:0+221
2014-08-22 18:04:22,836 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2008798054_0009_m_000000_0' done.
2014-08-22 18:04:22,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2008798054_0009_m_000000_0
2014-08-22 18:04:22,836 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:22,840 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@60ec0b80
2014-08-22 18:04:22,841 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,842 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:22,843 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 182 bytes
2014-08-22 18:04:22,844 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,844 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:22,844 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:22,857 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'num']]
2014-08-22 18:04:22,858 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['num', 'lower', 'upper']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstnfields"]
2014-08-22 18:04:22,866 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2008798054_0009_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:22,869 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:22,869 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2008798054_0009_r_000000_0 is allowed to commit now
2014-08-22 18:04:22,873 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2008798054_0009_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstnfields
2014-08-22 18:04:22,874 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:22,875 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2008798054_0009_r_000000_0' done.
2014-08-22 18:04:22,881 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/firstnfields/_temporary
2014-08-22 18:04:22,892 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,894 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/firstnfields
2014-08-22 18:04:22,908 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:22,908 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:22,930 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting
2014-08-22 18:04:22,931 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:22,931 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields"]
2014-08-22 18:04:22,931 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] parallel execution is enabled: false
2014-08-22 18:04:22,932 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting jobs: 1
2014-08-22 18:04:22,932 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] allocating threads: 1
2014-08-22 18:04:22,934 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...persplatform/coercefields
2014-08-22 18:04:22,945 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:22,971 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local316960116_0010
2014-08-22 18:04:22,972 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2014-08-22 18:04:22,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:22,975 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local316960116_0010_m_000000_0
2014-08-22 18:04:22,977 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1c898b41
2014-08-22 18:04:22,979 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:22,979 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@51cfc277
2014-08-22 18:04:22,981 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:22,985 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:22,985 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:22,994 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:22,994 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields"]
2014-08-22 18:04:23,003 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local316960116_0010_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,005 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,005 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local316960116_0010_m_000000_0 is allowed to commit now
2014-08-22 18:04:23,008 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local316960116_0010_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields
2014-08-22 18:04:23,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:23,010 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local316960116_0010_m_000000_0' done.
2014-08-22 18:04:23,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local316960116_0010_m_000000_0
2014-08-22 18:04:23,010 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:23,024 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coercefields/_temporary
2014-08-22 18:04:23,034 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,036 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/coercefields
2014-08-22 18:04:23,049 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:23,049 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:23,067 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-08-22 18:04:23,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count"]
2014-08-22 18:04:23,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-08-22 18:04:23,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-08-22 18:04:23,068 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-08-22 18:04:23,069 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...mblyhelpersplatform/count
2014-08-22 18:04:23,080 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,105 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1984193050_0011
2014-08-22 18:04:23,106 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-08-22 18:04:23,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:23,108 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1984193050_0011_m_000000_0
2014-08-22 18:04:23,110 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@66176ccc
2014-08-22 18:04:23,111 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:23,112 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@28154ec5
2014-08-22 18:04:23,113 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:23,113 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:23,244 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:23,244 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:23,246 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:23,246 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,257 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,258 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{1}:'char']]
2014-08-22 18:04:23,259 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:23,260 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:23,260 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 104, total: 329, max: 672
2014-08-22 18:04:23,261 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 48% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,261 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:23,262 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 104, total: 329, max: 672
2014-08-22 18:04:23,262 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 48% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,262 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:23,263 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 104, total: 329, max: 672
2014-08-22 18:04:23,263 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 48% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,263 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:23,264 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 104, total: 329, max: 672
2014-08-22 18:04:23,264 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 48% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,265 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:23,267 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:23,269 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1984193050_0011_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,273 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:23,273 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1984193050_0011_m_000000_0' done.
2014-08-22 18:04:23,274 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1984193050_0011_m_000000_0
2014-08-22 18:04:23,274 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:23,278 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@782bbb7b
2014-08-22 18:04:23,278 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,279 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:23,280 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2014-08-22 18:04:23,280 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,280 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:23,280 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,289 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{1}:'char']]
2014-08-22 18:04:23,289 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count"]
2014-08-22 18:04:23,295 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1984193050_0011_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,296 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,296 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1984193050_0011_r_000000_0 is allowed to commit now
2014-08-22 18:04:23,299 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1984193050_0011_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count
2014-08-22 18:04:23,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:23,300 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1984193050_0011_r_000000_0' done.
2014-08-22 18:04:23,310 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/count/_temporary
2014-08-22 18:04:23,320 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,330 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,332 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/count
2014-08-22 18:04:23,344 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:23,344 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:23,362 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting
2014-08-22 18:04:23,362 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,362 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] sink: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby"]
2014-08-22 18:04:23,362 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] parallel execution is enabled: false
2014-08-22 18:04:23,363 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] starting jobs: 1
2014-08-22 18:04:23,363 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [max] allocating threads: 1
2014-08-22 18:04:23,363 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] starting step: (1/1) ...mblyhelpersplatform/maxby
2014-08-22 18:04:23,374 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,401 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] submitted hadoop job: job_local393431847_0012
2014-08-22 18:04:23,401 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [max] tracking url: http://localhost:8080/
2014-08-22 18:04:23,403 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:23,404 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local393431847_0012_m_000000_0
2014-08-22 18:04:23,406 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5a0d2bee
2014-08-22 18:04:23,408 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:23,408 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6099210f
2014-08-22 18:04:23,410 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:23,411 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:23,557 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:23,558 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:23,560 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:23,560 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,570 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,570 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(max)[by:[{1}:'char']]
2014-08-22 18:04:23,571 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:23,572 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:23,573 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 88, total: 424, max: 672
2014-08-22 18:04:23,573 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,574 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:23,575 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 88, total: 424, max: 672
2014-08-22 18:04:23,575 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,576 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:23,576 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 88, total: 424, max: 672
2014-08-22 18:04:23,577 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:23,578 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 88, total: 424, max: 672
2014-08-22 18:04:23,579 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 63% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,579 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:23,582 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:23,585 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local393431847_0012_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,587 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:23,588 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local393431847_0012_m_000000_0' done.
2014-08-22 18:04:23,588 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local393431847_0012_m_000000_0
2014-08-22 18:04:23,589 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:23,594 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@60b07af1
2014-08-22 18:04:23,595 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,597 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:23,598 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 155 bytes
2014-08-22 18:04:23,599 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,599 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:23,599 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,608 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(max)[by:[{1}:'char']]
2014-08-22 18:04:23,608 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'max']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby"]
2014-08-22 18:04:23,614 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local393431847_0012_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,616 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,616 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local393431847_0012_r_000000_0 is allowed to commit now
2014-08-22 18:04:23,619 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local393431847_0012_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby
2014-08-22 18:04:23,621 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:23,621 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local393431847_0012_r_000000_0' done.
2014-08-22 18:04:23,635 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/maxby/_temporary
2014-08-22 18:04:23,646 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,658 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/maxby
2014-08-22 18:04:23,672 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:23,672 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:23,689 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] starting
2014-08-22 18:04:23,689 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,690 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] sink: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby"]
2014-08-22 18:04:23,690 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] parallel execution is enabled: false
2014-08-22 18:04:23,690 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] starting jobs: 1
2014-08-22 18:04:23,690 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [min] allocating threads: 1
2014-08-22 18:04:23,692 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] starting step: (1/1) ...mblyhelpersplatform/minby
2014-08-22 18:04:23,702 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,731 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] submitted hadoop job: job_local104600899_0013
2014-08-22 18:04:23,732 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [min] tracking url: http://localhost:8080/
2014-08-22 18:04:23,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:23,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local104600899_0013_m_000000_0
2014-08-22 18:04:23,737 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6908b095
2014-08-22 18:04:23,739 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:23,739 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7f001ba5
2014-08-22 18:04:23,740 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:23,740 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:23,874 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:23,874 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:23,876 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:23,876 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,885 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,885 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(min)[by:[{1}:'char']]
2014-08-22 18:04:23,885 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:23,887 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:23,887 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 520, max: 672
2014-08-22 18:04:23,887 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 77% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,888 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:23,888 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 520, max: 672
2014-08-22 18:04:23,888 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 77% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,889 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:23,889 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 520, max: 672
2014-08-22 18:04:23,889 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 77% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,890 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:23,890 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 520, max: 672
2014-08-22 18:04:23,890 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 77% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:23,891 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:23,894 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:23,896 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local104600899_0013_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,899 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:23,900 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local104600899_0013_m_000000_0' done.
2014-08-22 18:04:23,900 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local104600899_0013_m_000000_0
2014-08-22 18:04:23,900 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:23,904 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@13d5e499
2014-08-22 18:04:23,905 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,906 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:23,907 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 155 bytes
2014-08-22 18:04:23,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,908 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:23,908 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:23,916 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(min)[by:[{1}:'char']]
2014-08-22 18:04:23,916 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'min']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby"]
2014-08-22 18:04:23,922 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local104600899_0013_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:23,924 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:23,924 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local104600899_0013_r_000000_0 is allowed to commit now
2014-08-22 18:04:23,927 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local104600899_0013_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby
2014-08-22 18:04:23,928 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:23,929 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local104600899_0013_r_000000_0' done.
2014-08-22 18:04:23,936 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/minby/_temporary
2014-08-22 18:04:23,947 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,960 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:23,961 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/minby
2014-08-22 18:04:23,975 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:23,975 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:23,994 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting
2014-08-22 18:04:23,994 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:23,995 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum"]
2014-08-22 18:04:23,995 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] parallel execution is enabled: false
2014-08-22 18:04:23,995 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting jobs: 1
2014-08-22 18:04:23,995 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] allocating threads: 1
2014-08-22 18:04:23,996 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...semblyhelpersplatform/sum
2014-08-22 18:04:24,007 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,038 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local1824793634_0014
2014-08-22 18:04:24,038 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2014-08-22 18:04:24,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:24,041 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1824793634_0014_m_000000_0
2014-08-22 18:04:24,043 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1cfb765a
2014-08-22 18:04:24,045 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:24,046 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@67a88328
2014-08-22 18:04:24,047 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:24,048 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:24,194 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:24,196 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:24,197 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:24,198 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,208 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:24,209 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2014-08-22 18:04:24,209 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:24,211 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:24,211 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 97, total: 587, max: 672
2014-08-22 18:04:24,212 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:24,212 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 97, total: 587, max: 672
2014-08-22 18:04:24,213 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:24,213 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 97, total: 587, max: 672
2014-08-22 18:04:24,214 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:24,214 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 97, total: 587, max: 672
2014-08-22 18:04:24,214 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:24,217 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:24,219 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1824793634_0014_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:24,221 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:24,222 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1824793634_0014_m_000000_0' done.
2014-08-22 18:04:24,222 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1824793634_0014_m_000000_0
2014-08-22 18:04:24,222 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:24,226 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7ce229be
2014-08-22 18:04:24,226 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,227 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:24,227 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2014-08-22 18:04:24,228 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,228 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:24,229 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,236 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2014-08-22 18:04:24,236 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum"]
2014-08-22 18:04:24,242 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1824793634_0014_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:24,244 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,244 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1824793634_0014_r_000000_0 is allowed to commit now
2014-08-22 18:04:24,246 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1824793634_0014_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum
2014-08-22 18:04:24,248 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:24,248 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1824793634_0014_r_000000_0' done.
2014-08-22 18:04:24,256 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sum/_temporary
2014-08-22 18:04:24,265 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,276 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,277 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/sum
2014-08-22 18:04:24,291 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:24,291 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:24,312 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] starting
2014-08-22 18:04:24,312 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:24,313 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:24,313 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum"]
2014-08-22 18:04:24,313 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] parallel execution is enabled: false
2014-08-22 18:04:24,313 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] starting jobs: 1
2014-08-22 18:04:24,313 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum-lhs+sum-rhs] allocating threads: 1
2014-08-22 18:04:24,314 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] starting step: (1/1) ...yhelpersplatform/mergesum
2014-08-22 18:04:24,324 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,325 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,349 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] submitted hadoop job: job_local1326171192_0015
2014-08-22 18:04:24,350 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum-lhs+sum-rhs] tracking url: http://localhost:8080/
2014-08-22 18:04:24,352 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:24,352 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1326171192_0015_m_000000_0
2014-08-22 18:04:24,354 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@305f6ed4
2014-08-22 18:04:24,356 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:24,357 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6469adc7
2014-08-22 18:04:24,358 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:24,359 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:24,655 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:24,656 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:24,657 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:24,657 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,668 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:24,668 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-08-22 18:04:24,668 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:24,670 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:24,670 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 79, total: 184, max: 672
2014-08-22 18:04:24,670 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 27% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:24,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 79, total: 184, max: 672
2014-08-22 18:04:24,671 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 27% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:24,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 79, total: 184, max: 672
2014-08-22 18:04:24,672 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 27% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,673 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:24,673 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 79, total: 184, max: 672
2014-08-22 18:04:24,673 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 27% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,674 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:24,677 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:24,679 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1326171192_0015_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:24,681 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:24,681 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1326171192_0015_m_000000_0' done.
2014-08-22 18:04:24,682 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1326171192_0015_m_000000_0
2014-08-22 18:04:24,682 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1326171192_0015_m_000001_0
2014-08-22 18:04:24,683 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@cf9b31d
2014-08-22 18:04:24,685 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:24,685 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@271ff1b8
2014-08-22 18:04:24,686 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:24,687 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:24,757 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:24,757 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:24,759 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:24,759 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,769 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:24,770 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-08-22 18:04:24,771 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:24,774 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:24,775 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 279, max: 672
2014-08-22 18:04:24,775 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 41% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,776 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:24,776 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 279, max: 672
2014-08-22 18:04:24,776 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 41% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,777 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:24,777 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 279, max: 672
2014-08-22 18:04:24,778 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 41% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,778 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:24,778 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 72, total: 279, max: 672
2014-08-22 18:04:24,779 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 41% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:24,779 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:24,786 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:24,788 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1326171192_0015_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:24,790 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:24,791 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1326171192_0015_m_000001_0' done.
2014-08-22 18:04:24,791 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1326171192_0015_m_000001_0
2014-08-22 18:04:24,791 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:24,799 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6de9b48b
2014-08-22 18:04:24,799 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,800 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:24,801 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 238 bytes
2014-08-22 18:04:24,802 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,802 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:24,802 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,811 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum-lhs+sum-rhs)[by: sum-lhs:[{1}:'char'] sum-rhs:[{1}:'char']]
2014-08-22 18:04:24,812 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum"]
2014-08-22 18:04:24,819 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1326171192_0015_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:24,821 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,821 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1326171192_0015_r_000000_0 is allowed to commit now
2014-08-22 18:04:24,824 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1326171192_0015_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum
2014-08-22 18:04:24,825 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:24,825 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1326171192_0015_r_000000_0' done.
2014-08-22 18:04:24,840 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/mergesum/_temporary
2014-08-22 18:04:24,851 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,863 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,865 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/mergesum
2014-08-22 18:04:24,877 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:24,878 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:24,893 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:24,893 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:24,893 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow"]
2014-08-22 18:04:24,893 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:24,894 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:24,894 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:24,894 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...ersplatform/discardnarrow
2014-08-22 18:04:24,904 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,942 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local1618629398_0016
2014-08-22 18:04:24,943 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:24,945 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:24,945 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1618629398_0016_m_000000_0
2014-08-22 18:04:24,946 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3c954549
2014-08-22 18:04:24,948 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:24,948 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7da79447
2014-08-22 18:04:24,949 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:24,952 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:24,952 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:24,957 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:24,957 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow"]
2014-08-22 18:04:24,959 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1618629398_0016_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:24,961 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:24,961 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1618629398_0016_m_000000_0 is allowed to commit now
2014-08-22 18:04:24,963 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1618629398_0016_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow
2014-08-22 18:04:24,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:24,965 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1618629398_0016_m_000000_0' done.
2014-08-22 18:04:24,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1618629398_0016_m_000000_0
2014-08-22 18:04:24,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:24,975 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/discardnarrow/_temporary
2014-08-22 18:04:24,984 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:24,985 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/discardnarrow
2014-08-22 18:04:24,996 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:24,997 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:25,007 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:25,007 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:25,008 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename"]
2014-08-22 18:04:25,008 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:25,008 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:25,008 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:25,009 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/rename
2014-08-22 18:04:25,017 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,042 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local2020840728_0017
2014-08-22 18:04:25,042 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:25,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:25,045 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2020840728_0017_m_000000_0
2014-08-22 18:04:25,047 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1361c602
2014-08-22 18:04:25,048 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:25,049 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@3301b455
2014-08-22 18:04:25,050 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:25,053 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:25,053 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,058 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:25,059 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['item', 'element']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename"]
2014-08-22 18:04:25,060 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2020840728_0017_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,062 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,063 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2020840728_0017_m_000000_0 is allowed to commit now
2014-08-22 18:04:25,065 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2020840728_0017_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename
2014-08-22 18:04:25,067 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:25,067 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2020840728_0017_m_000000_0' done.
2014-08-22 18:04:25,067 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2020840728_0017_m_000000_0
2014-08-22 18:04:25,068 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:25,075 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/rename/_temporary
2014-08-22 18:04:25,084 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,086 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/rename
2014-08-22 18:04:25,101 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:25,101 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:25,126 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-08-22 18:04:25,126 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:25,127 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:25,127 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true"]
2014-08-22 18:04:25,127 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] parallel execution is enabled: false
2014-08-22 18:04:25,127 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting jobs: 1
2014-08-22 18:04:25,127 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] allocating threads: 1
2014-08-22 18:04:25,128 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...splatform/multimerge+true
2014-08-22 18:04:25,137 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,138 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,161 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local525993968_0018
2014-08-22 18:04:25,162 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-08-22 18:04:25,164 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:25,164 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local525993968_0018_m_000000_0
2014-08-22 18:04:25,166 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4d29efd0
2014-08-22 18:04:25,167 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:25,167 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1bdbbaab
2014-08-22 18:04:25,169 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:25,169 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:25,459 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:25,459 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:25,460 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:25,461 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,472 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:25,472 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-08-22 18:04:25,473 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:25,474 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:25,475 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 275, total: 379, max: 672
2014-08-22 18:04:25,475 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,476 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:25,476 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 275, total: 379, max: 672
2014-08-22 18:04:25,477 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,478 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:25,478 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 275, total: 379, max: 672
2014-08-22 18:04:25,478 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,479 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:25,480 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 275, total: 379, max: 672
2014-08-22 18:04:25,480 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,481 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:25,483 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:25,485 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local525993968_0018_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,487 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:25,488 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local525993968_0018_m_000000_0' done.
2014-08-22 18:04:25,488 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local525993968_0018_m_000000_0
2014-08-22 18:04:25,488 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local525993968_0018_m_000001_0
2014-08-22 18:04:25,490 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3329aa21
2014-08-22 18:04:25,491 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:25,491 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4037c7ee
2014-08-22 18:04:25,493 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:25,493 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:25,511 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:25,511 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:25,513 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:25,513 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,524 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:25,524 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-08-22 18:04:25,525 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:25,529 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:25,529 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 173, total: 379, max: 672
2014-08-22 18:04:25,530 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,531 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:25,532 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 173, total: 379, max: 672
2014-08-22 18:04:25,532 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,533 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:25,533 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 173, total: 379, max: 672
2014-08-22 18:04:25,534 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,535 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:25,535 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 173, total: 379, max: 672
2014-08-22 18:04:25,535 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,536 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:25,539 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:25,540 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local525993968_0018_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:25,543 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:25,543 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local525993968_0018_m_000001_0' done.
2014-08-22 18:04:25,543 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local525993968_0018_m_000001_0
2014-08-22 18:04:25,544 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:25,550 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1e085b8d
2014-08-22 18:04:25,550 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,551 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:25,552 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 472 bytes
2014-08-22 18:04:25,552 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,553 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:25,553 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,563 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2014-08-22 18:04:25,563 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true"]
2014-08-22 18:04:25,570 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local525993968_0018_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,572 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,572 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local525993968_0018_r_000000_0 is allowed to commit now
2014-08-22 18:04:25,574 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local525993968_0018_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true
2014-08-22 18:04:25,576 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:25,576 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local525993968_0018_r_000000_0' done.
2014-08-22 18:04:25,582 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multimerge+true/_temporary
2014-08-22 18:04:25,591 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,601 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,603 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multimerge+true
2014-08-22 18:04:25,618 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:25,618 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:25,651 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] starting
2014-08-22 18:04:25,651 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:25,652 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount"]
2014-08-22 18:04:25,652 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] parallel execution is enabled: false
2014-08-22 18:04:25,652 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] starting jobs: 2
2014-08-22 18:04:25,652 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [second] allocating threads: 1
2014-08-22 18:04:25,652 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (1/2)
2014-08-22 18:04:25,661 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,684 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local754236778_0019
2014-08-22 18:04:25,684 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2014-08-22 18:04:25,686 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:25,687 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local754236778_0019_m_000000_0
2014-08-22 18:04:25,688 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@5cf18070
2014-08-22 18:04:25,689 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:25,689 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@733c0466
2014-08-22 18:04:25,690 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:25,691 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:25,710 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:25,710 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:25,711 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:25,711 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,720 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:25,720 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(first)[by:[{1}:'char']]
2014-08-22 18:04:25,721 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:25,724 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:25,725 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 379, max: 672
2014-08-22 18:04:25,725 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,726 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:25,726 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 379, max: 672
2014-08-22 18:04:25,726 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,727 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:25,727 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 379, max: 672
2014-08-22 18:04:25,727 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,728 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:25,728 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 379, max: 672
2014-08-22 18:04:25,728 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,729 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:25,732 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:25,733 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local754236778_0019_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,735 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:25,736 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local754236778_0019_m_000000_0' done.
2014-08-22 18:04:25,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local754236778_0019_m_000000_0
2014-08-22 18:04:25,736 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:25,739 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@455ef45a
2014-08-22 18:04:25,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,741 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:25,741 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2014-08-22 18:04:25,742 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,742 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:25,742 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,751 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(first)[by:[{1}:'char']]
2014-08-22 18:04:25,751 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5376386327/first/]
2014-08-22 18:04:25,756 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local754236778_0019_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,758 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:25,758 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local754236778_0019_r_000000_0 is allowed to commit now
2014-08-22 18:04:25,760 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local754236778_0019_r_000000_0' to file:/tmp/hadoop-teamcity/5376386327_first_6D02851D11C94F128DA9F21A2EA119AB
2014-08-22 18:04:25,761 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:25,761 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local754236778_0019_r_000000_0' done.
2014-08-22 18:04:25,768 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] starting step: (2/2) ...form/samesourcemergecount
2014-08-22 18:04:25,776 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:25,805 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] submitted hadoop job: job_local660216734_0020
2014-08-22 18:04:25,808 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [second] tracking url: http://localhost:8080/
2014-08-22 18:04:25,809 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:25,810 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local660216734_0020_m_000000_0
2014-08-22 18:04:25,813 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@895772a
2014-08-22 18:04:25,815 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5376386327_first_6D02851D11C94F128DA9F21A2EA119AB/part-00000
2014-08-22 18:04:25,816 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6eb1a0f9
2014-08-22 18:04:25,818 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:25,819 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:25,969 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:25,970 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:25,973 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:25,974 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:25,986 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5376386327/first/]
2014-08-22 18:04:25,986 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(second)[by:[{1}:'char']]
2014-08-22 18:04:25,987 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:25,988 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:25,988 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 46, total: 474, max: 672
2014-08-22 18:04:25,989 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 70% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,989 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:25,990 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 46, total: 474, max: 672
2014-08-22 18:04:25,990 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 70% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:25,991 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:25,993 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:25,995 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local660216734_0020_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:25,998 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/5376386327_first_6D02851D11C94F128DA9F21A2EA119AB/part-00000:0+160
2014-08-22 18:04:25,998 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local660216734_0020_m_000000_0' done.
2014-08-22 18:04:25,998 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local660216734_0020_m_000000_0
2014-08-22 18:04:25,999 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:26,003 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@d1b9cfd
2014-08-22 18:04:26,004 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,005 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:26,005 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2014-08-22 18:04:26,006 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,007 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:26,007 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,019 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(second)[by:[{1}:'char']]
2014-08-22 18:04:26,020 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount"]
2014-08-22 18:04:26,025 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local660216734_0020_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:26,027 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,027 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local660216734_0020_r_000000_0 is allowed to commit now
2014-08-22 18:04:26,030 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local660216734_0020_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount
2014-08-22 18:04:26,031 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:26,031 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local660216734_0020_r_000000_0' done.
2014-08-22 18:04:26,043 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergecount/_temporary
2014-08-22 18:04:26,053 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,065 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,067 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/samesourcemergecount
2014-08-22 18:04:26,079 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:26,080 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:26,093 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting
2014-08-22 18:04:26,093 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,093 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average"]
2014-08-22 18:04:26,094 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] parallel execution is enabled: false
2014-08-22 18:04:26,094 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting jobs: 1
2014-08-22 18:04:26,094 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] allocating threads: 1
2014-08-22 18:04:26,095 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lyhelpersplatform/average
2014-08-22 18:04:26,105 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,135 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1367723155_0021
2014-08-22 18:04:26,135 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2014-08-22 18:04:26,137 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:26,138 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1367723155_0021_m_000000_0
2014-08-22 18:04:26,139 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7d91f64b
2014-08-22 18:04:26,141 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:26,141 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@7b04eff0
2014-08-22 18:04:26,142 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:26,142 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:26,279 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:26,279 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:26,280 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:26,280 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,288 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,288 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2014-08-22 18:04:26,289 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:26,290 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:26,290 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 29, total: 570, max: 672
2014-08-22 18:04:26,291 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:26,291 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 29, total: 570, max: 672
2014-08-22 18:04:26,292 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:26,292 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 29, total: 570, max: 672
2014-08-22 18:04:26,293 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:26,293 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 29, total: 570, max: 672
2014-08-22 18:04:26,294 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:26,296 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:26,298 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1367723155_0021_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:26,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:26,300 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1367723155_0021_m_000000_0' done.
2014-08-22 18:04:26,300 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1367723155_0021_m_000000_0
2014-08-22 18:04:26,301 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:26,304 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7792a63a
2014-08-22 18:04:26,305 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,306 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:26,306 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 200 bytes
2014-08-22 18:04:26,307 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,307 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:26,307 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,314 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2014-08-22 18:04:26,314 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average"]
2014-08-22 18:04:26,320 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1367723155_0021_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:26,321 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,322 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1367723155_0021_r_000000_0 is allowed to commit now
2014-08-22 18:04:26,324 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1367723155_0021_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average
2014-08-22 18:04:26,325 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:26,325 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1367723155_0021_r_000000_0' done.
2014-08-22 18:04:26,340 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/average/_temporary
2014-08-22 18:04:26,351 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,363 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,364 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/average
2014-08-22 18:04:26,378 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:26,379 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:26,393 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting
2014-08-22 18:04:26,394 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,394 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] sink: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls"]
2014-08-22 18:04:26,394 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] parallel execution is enabled: false
2014-08-22 18:04:26,394 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] starting jobs: 1
2014-08-22 18:04:26,394 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [sum] allocating threads: 1
2014-08-22 18:04:26,395 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] starting step: (1/1) ...yhelpersplatform/sumnulls
2014-08-22 18:04:26,406 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,435 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] submitted hadoop job: job_local962736275_0022
2014-08-22 18:04:26,436 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [sum] tracking url: http://localhost:8080/
2014-08-22 18:04:26,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:26,438 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local962736275_0022_m_000000_0
2014-08-22 18:04:26,440 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3cc0843d
2014-08-22 18:04:26,441 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:26,441 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@11d612fc
2014-08-22 18:04:26,443 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:26,443 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:26,580 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:26,580 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:26,581 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:26,582 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,589 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,589 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(sum)[by:[{1}:'char']]
2014-08-22 18:04:26,590 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:26,597 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:26,597 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 11, total: 665, max: 672
2014-08-22 18:04:26,598 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:26,598 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 11, total: 665, max: 672
2014-08-22 18:04:26,599 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:26,599 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 11, total: 665, max: 672
2014-08-22 18:04:26,600 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:26,600 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 11, total: 665, max: 672
2014-08-22 18:04:26,601 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:26,603 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:26,605 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local962736275_0022_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:26,607 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:26,607 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local962736275_0022_m_000000_0' done.
2014-08-22 18:04:26,607 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local962736275_0022_m_000000_0
2014-08-22 18:04:26,607 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:26,611 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@54737611
2014-08-22 18:04:26,612 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,612 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:26,613 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 116 bytes
2014-08-22 18:04:26,613 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,614 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:26,614 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,620 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(sum)[by:[{1}:'char']]
2014-08-22 18:04:26,620 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls"]
2014-08-22 18:04:26,625 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local962736275_0022_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:26,627 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:26,627 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local962736275_0022_r_000000_0 is allowed to commit now
2014-08-22 18:04:26,630 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local962736275_0022_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls
2014-08-22 18:04:26,631 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:26,631 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local962736275_0022_r_000000_0' done.
2014-08-22 18:04:26,636 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/sumnulls/_temporary
2014-08-22 18:04:26,646 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,658 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/sumnulls
2014-08-22 18:04:26,913 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:26,914 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:26,925 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-08-22 18:04:26,925 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,925 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall"]
2014-08-22 18:04:26,926 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-08-22 18:04:26,926 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-08-22 18:04:26,926 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-08-22 18:04:26,927 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...yhelpersplatform/countall
2014-08-22 18:04:26,935 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:26,959 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local85273019_0023
2014-08-22 18:04:26,961 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-08-22 18:04:26,964 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:26,965 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local85273019_0023_m_000000_0
2014-08-22 18:04:26,966 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@76996f0c
2014-08-22 18:04:26,967 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:26,967 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@25deb1ad
2014-08-22 18:04:26,968 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:26,969 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:26,986 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:26,986 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:26,988 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:26,988 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:26,995 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:26,995 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2014-08-22 18:04:26,996 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:26,998 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:27,000 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:27,001 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local85273019_0023_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,003 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:27,003 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local85273019_0023_m_000000_0' done.
2014-08-22 18:04:27,004 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local85273019_0023_m_000000_0
2014-08-22 18:04:27,004 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,007 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6c506ee7
2014-08-22 18:04:27,008 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,008 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:27,009 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 8 bytes
2014-08-22 18:04:27,009 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,010 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:27,010 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,017 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2014-08-22 18:04:27,017 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall"]
2014-08-22 18:04:27,021 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local85273019_0023_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,023 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,023 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local85273019_0023_r_000000_0 is allowed to commit now
2014-08-22 18:04:27,025 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local85273019_0023_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall
2014-08-22 18:04:27,026 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:27,026 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local85273019_0023_r_000000_0' done.
2014-08-22 18:04:27,034 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countall/_temporary
2014-08-22 18:04:27,042 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,052 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,053 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countall
2014-08-22 18:04:27,066 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,067 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,081 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting
2014-08-22 18:04:27,082 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:27,082 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] sink: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi"]
2014-08-22 18:04:27,082 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] parallel execution is enabled: false
2014-08-22 18:04:27,082 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] starting jobs: 1
2014-08-22 18:04:27,082 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [name] allocating threads: 1
2014-08-22 18:04:27,083 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] starting step: (1/1) ...mblyhelpersplatform/multi
2014-08-22 18:04:27,091 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,116 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] submitted hadoop job: job_local529550439_0024
2014-08-22 18:04:27,117 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [name] tracking url: http://localhost:8080/
2014-08-22 18:04:27,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:27,119 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local529550439_0024_m_000000_0
2014-08-22 18:04:27,120 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@75896415
2014-08-22 18:04:27,121 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:27,121 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5769aa0e
2014-08-22 18:04:27,122 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:27,123 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:27,141 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:27,142 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:27,144 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:27,144 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,153 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:27,153 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(name)[by:[{1}:'char']]
2014-08-22 18:04:27,154 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:27,155 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:27,155 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 60, total: 286, max: 672
2014-08-22 18:04:27,155 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 42% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,157 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:27,157 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 60, total: 286, max: 672
2014-08-22 18:04:27,157 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 42% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,158 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:27,158 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 60, total: 286, max: 672
2014-08-22 18:04:27,159 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 42% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,160 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:27,160 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 59, total: 286, max: 672
2014-08-22 18:04:27,160 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 42% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,161 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:27,163 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:27,164 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local529550439_0024_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,166 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:27,166 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local529550439_0024_m_000000_0' done.
2014-08-22 18:04:27,167 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local529550439_0024_m_000000_0
2014-08-22 18:04:27,167 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,170 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@acc0596
2014-08-22 18:04:27,171 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,172 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:27,172 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 452 bytes
2014-08-22 18:04:27,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,175 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:27,175 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,185 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(name)[by:[{1}:'char']]
2014-08-22 18:04:27,185 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'sum', 'count', 'average', 'average2', 'first']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi"]
2014-08-22 18:04:27,191 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local529550439_0024_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,193 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,193 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local529550439_0024_r_000000_0 is allowed to commit now
2014-08-22 18:04:27,195 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local529550439_0024_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi
2014-08-22 18:04:27,196 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:27,197 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local529550439_0024_r_000000_0' done.
2014-08-22 18:04:27,210 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/multi/_temporary
2014-08-22 18:04:27,219 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,229 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,231 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/multi
2014-08-22 18:04:27,240 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,241 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,250 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:27,251 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,251 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow"]
2014-08-22 18:04:27,251 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:27,251 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:27,251 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:27,252 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/retainnarrow
2014-08-22 18:04:27,260 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,281 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local895214179_0025
2014-08-22 18:04:27,282 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:27,284 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:27,284 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local895214179_0025_m_000000_0
2014-08-22 18:04:27,285 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@580a5adc
2014-08-22 18:04:27,286 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:27,287 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@220860ba
2014-08-22 18:04:27,288 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:27,290 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:27,291 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,295 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,295 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['num']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow"]
2014-08-22 18:04:27,297 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local895214179_0025_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,298 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,298 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local895214179_0025_m_000000_0 is allowed to commit now
2014-08-22 18:04:27,301 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local895214179_0025_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow
2014-08-22 18:04:27,302 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:27,302 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local895214179_0025_m_000000_0' done.
2014-08-22 18:04:27,302 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local895214179_0025_m_000000_0
2014-08-22 18:04:27,302 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,314 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/retainnarrow/_temporary
2014-08-22 18:04:27,323 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,324 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/retainnarrow
2014-08-22 18:04:27,333 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,334 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,342 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:27,342 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,343 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow"]
2014-08-22 18:04:27,343 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:27,343 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:27,343 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:27,343 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...persplatform/renamenarrow
2014-08-22 18:04:27,351 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,371 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local2064600457_0026
2014-08-22 18:04:27,371 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:27,373 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:27,374 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2064600457_0026_m_000000_0
2014-08-22 18:04:27,375 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@9d451d2
2014-08-22 18:04:27,376 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:27,376 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@1617e649
2014-08-22 18:04:27,377 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:27,380 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:27,380 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,384 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,385 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['item']->['char', 'item']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow"]
2014-08-22 18:04:27,387 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2064600457_0026_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,389 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,389 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2064600457_0026_m_000000_0 is allowed to commit now
2014-08-22 18:04:27,391 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2064600457_0026_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow
2014-08-22 18:04:27,392 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:27,392 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2064600457_0026_m_000000_0' done.
2014-08-22 18:04:27,393 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2064600457_0026_m_000000_0
2014-08-22 18:04:27,393 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,403 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/renamenarrow/_temporary
2014-08-22 18:04:27,412 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,413 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/renamenarrow
2014-08-22 18:04:27,424 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,425 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,435 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting
2014-08-22 18:04:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] sink: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull"]
2014-08-22 18:04:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] parallel execution is enabled: false
2014-08-22 18:04:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] starting jobs: 1
2014-08-22 18:04:27,436 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [average] allocating threads: 1
2014-08-22 18:04:27,437 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] starting step: (1/1) ...lpersplatform/averagenull
2014-08-22 18:04:27,445 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,470 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] submitted hadoop job: job_local1688689389_0027
2014-08-22 18:04:27,470 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [average] tracking url: http://localhost:8080/
2014-08-22 18:04:27,473 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:27,473 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1688689389_0027_m_000000_0
2014-08-22 18:04:27,475 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@16912f10
2014-08-22 18:04:27,476 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:27,476 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5efa5e8f
2014-08-22 18:04:27,478 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:27,478 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:27,614 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:27,614 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:27,615 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:27,616 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,624 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:27,624 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(average)[by:[{1}:'char']]
2014-08-22 18:04:27,625 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:27,629 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:27,629 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 22, total: 381, max: 672
2014-08-22 18:04:27,629 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,630 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:27,630 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 22, total: 381, max: 672
2014-08-22 18:04:27,631 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,631 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:27,632 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 22, total: 381, max: 672
2014-08-22 18:04:27,632 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,632 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:27,633 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 22, total: 381, max: 672
2014-08-22 18:04:27,633 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 56% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:27,633 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:27,636 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:27,637 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1688689389_0027_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,639 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:27,639 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1688689389_0027_m_000000_0' done.
2014-08-22 18:04:27,639 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1688689389_0027_m_000000_0
2014-08-22 18:04:27,640 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,643 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@53911ec8
2014-08-22 18:04:27,643 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,645 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:27,646 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 200 bytes
2014-08-22 18:04:27,646 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,647 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:27,647 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,654 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(average)[by:[{1}:'char']]
2014-08-22 18:04:27,654 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'average']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull"]
2014-08-22 18:04:27,660 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1688689389_0027_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,661 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,661 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1688689389_0027_r_000000_0 is allowed to commit now
2014-08-22 18:04:27,663 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1688689389_0027_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull
2014-08-22 18:04:27,664 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:27,665 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1688689389_0027_r_000000_0' done.
2014-08-22 18:04:27,674 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/averagenull/_temporary
2014-08-22 18:04:27,683 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,693 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,694 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/averagenull
2014-08-22 18:04:27,704 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,704 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,712 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting
2014-08-22 18:04:27,713 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,713 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] sink: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce"]
2014-08-22 18:04:27,713 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] parallel execution is enabled: false
2014-08-22 18:04:27,713 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] starting jobs: 1
2014-08-22 18:04:27,713 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [coerce] allocating threads: 1
2014-08-22 18:04:27,714 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] starting step: (1/1) ...blyhelpersplatform/coerce
2014-08-22 18:04:27,722 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,744 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] submitted hadoop job: job_local475620900_0028
2014-08-22 18:04:27,744 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [coerce] tracking url: http://localhost:8080/
2014-08-22 18:04:27,746 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:27,878 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local475620900_0028_m_000000_0
2014-08-22 18:04:27,880 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@34471b39
2014-08-22 18:04:27,881 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt
2014-08-22 18:04:27,882 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5d39af80
2014-08-22 18:04:27,883 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 0
2014-08-22 18:04:27,885 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:27,885 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:27,889 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt"]
2014-08-22 18:04:27,890 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce"]
2014-08-22 18:04:27,891 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local475620900_0028_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:27,893 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:27,893 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local475620900_0028_m_000000_0 is allowed to commit now
2014-08-22 18:04:27,895 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local475620900_0028_m_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce
2014-08-22 18:04:27,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lower.txt:0+19
2014-08-22 18:04:27,896 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local475620900_0028_m_000000_0' done.
2014-08-22 18:04:27,896 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local475620900_0028_m_000000_0
2014-08-22 18:04:27,897 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:27,910 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/coerce/_temporary
2014-08-22 18:04:27,918 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:27,920 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/coerce
2014-08-22 18:04:27,936 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:27,936 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:27,985 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] starting
2014-08-22 18:04:27,986 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:27,987 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] sink: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway"]
2014-08-22 18:04:27,988 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] parallel execution is enabled: false
2014-08-22 18:04:27,988 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] starting jobs: 5
2014-08-22 18:04:27,989 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count-lhs*source*count...] allocating threads: 1
2014-08-22 18:04:27,991 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (1/5)
2014-08-22 18:04:27,999 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,021 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1265859360_0029
2014-08-22 18:04:28,022 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-08-22 18:04:28,023 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:28,024 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1265859360_0029_m_000000_0
2014-08-22 18:04:28,025 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4a8b88bd
2014-08-22 18:04:28,026 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:28,026 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@68c6fc84
2014-08-22 18:04:28,027 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,027 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,045 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,046 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,047 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,047 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,054 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:28,054 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(source)[by:[{1}:'char']]
2014-08-22 18:04:28,055 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:28,056 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:28,056 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 170, total: 290, max: 672
2014-08-22 18:04:28,057 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,060 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:28,060 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 170, total: 290, max: 672
2014-08-22 18:04:28,060 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,061 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 5, with threshold: 2
2014-08-22 18:04:28,062 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 170, total: 290, max: 672
2014-08-22 18:04:28,062 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,063 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 7, with threshold: 2
2014-08-22 18:04:28,063 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 170, total: 290, max: 672
2014-08-22 18:04:28,063 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,064 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,066 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,068 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1265859360_0029_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,070 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:28,070 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1265859360_0029_m_000000_0' done.
2014-08-22 18:04:28,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1265859360_0029_m_000000_0
2014-08-22 18:04:28,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:28,074 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7df36af6
2014-08-22 18:04:28,075 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,075 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:28,076 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 119 bytes
2014-08-22 18:04:28,076 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,077 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:28,077 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,082 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(source)[by:[{1}:'char']]
2014-08-22 18:04:28,082 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][461280548/source/]
2014-08-22 18:04:28,087 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1265859360_0029_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,088 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,088 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1265859360_0029_r_000000_0 is allowed to commit now
2014-08-22 18:04:28,090 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1265859360_0029_r_000000_0' to file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59
2014-08-22 18:04:28,091 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:28,091 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1265859360_0029_r_000000_0' done.
2014-08-22 18:04:28,095 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (3/5)
2014-08-22 18:04:28,103 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,127 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1267473938_0030
2014-08-22 18:04:28,127 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-08-22 18:04:28,129 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:28,129 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1267473938_0030_m_000000_0
2014-08-22 18:04:28,130 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@c74cc70
2014-08-22 18:04:28,131 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000
2014-08-22 18:04:28,131 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6ae03500
2014-08-22 18:04:28,133 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,134 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,152 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,153 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,154 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,154 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,161 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][461280548/source/]
2014-08-22 18:04:28,161 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-lhs)[by:[{1}:'char']]
2014-08-22 18:04:28,162 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:28,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:28,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 63, total: 290, max: 672
2014-08-22 18:04:28,163 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:28,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 63, total: 290, max: 672
2014-08-22 18:04:28,164 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 43% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,164 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,166 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,168 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1267473938_0030_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,169 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000:0+160
2014-08-22 18:04:28,170 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1267473938_0030_m_000000_0' done.
2014-08-22 18:04:28,170 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1267473938_0030_m_000000_0
2014-08-22 18:04:28,170 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:28,173 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3af036de
2014-08-22 18:04:28,174 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,174 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:28,175 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2014-08-22 18:04:28,175 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,175 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:28,176 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,180 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-lhs)[by:[{1}:'char']]
2014-08-22 18:04:28,181 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][5441208656/count-lhs/]
2014-08-22 18:04:28,184 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1267473938_0030_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,186 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,186 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1267473938_0030_r_000000_0 is allowed to commit now
2014-08-22 18:04:28,188 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1267473938_0030_r_000000_0' to file:/tmp/hadoop-teamcity/5441208656_count_lhs_22DA6773C85A46AAAFD146FC04030CBC
2014-08-22 18:04:28,189 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:28,189 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1267473938_0030_r_000000_0' done.
2014-08-22 18:04:28,200 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (4/5)
2014-08-22 18:04:28,207 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,231 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local992116102_0031
2014-08-22 18:04:28,232 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-08-22 18:04:28,233 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:28,234 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local992116102_0031_m_000000_0
2014-08-22 18:04:28,235 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@251486c5
2014-08-22 18:04:28,236 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000
2014-08-22 18:04:28,236 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@587fc8a0
2014-08-22 18:04:28,237 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,238 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,362 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,362 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,364 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,369 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][461280548/source/]
2014-08-22 18:04:28,369 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count-rhs)[by:[{1}:'char']]
2014-08-22 18:04:28,370 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:28,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-08-22 18:04:28,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 51, total: 385, max: 672
2014-08-22 18:04:28,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,371 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-08-22 18:04:28,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 51, total: 385, max: 672
2014-08-22 18:04:28,372 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 57% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-08-22 18:04:28,372 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,374 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,375 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local992116102_0031_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,377 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000:0+160
2014-08-22 18:04:28,377 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local992116102_0031_m_000000_0' done.
2014-08-22 18:04:28,378 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local992116102_0031_m_000000_0
2014-08-22 18:04:28,378 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:28,381 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@87a1904
2014-08-22 18:04:28,381 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,382 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:28,382 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 67 bytes
2014-08-22 18:04:28,383 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,383 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:28,383 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,390 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count-rhs)[by:[{1}:'char']]
2014-08-22 18:04:28,390 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'count']]"][6552556201/count-rhs/]
2014-08-22 18:04:28,394 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local992116102_0031_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,395 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,395 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local992116102_0031_r_000000_0 is allowed to commit now
2014-08-22 18:04:28,397 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local992116102_0031_r_000000_0' to file:/tmp/hadoop-teamcity/6552556201_count_rhs_2D4F13B5CC864BCCB5494D78AE38AB23
2014-08-22 18:04:28,398 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:28,398 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local992116102_0031_r_000000_0' done.
2014-08-22 18:04:28,406 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (2/5)
2014-08-22 18:04:28,415 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,416 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,437 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1347302405_0032
2014-08-22 18:04:28,438 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-08-22 18:04:28,439 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:28,439 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1347302405_0032_m_000000_0
2014-08-22 18:04:28,440 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@6162ae62
2014-08-22 18:04:28,441 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/6552556201_count_rhs_2D4F13B5CC864BCCB5494D78AE38AB23/part-00000
2014-08-22 18:04:28,442 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5004f41c
2014-08-22 18:04:28,443 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,443 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,576 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,576 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,578 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,578 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,584 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][6552556201/count-rhs/]
2014-08-22 18:04:28,585 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:28,587 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,588 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,590 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1347302405_0032_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/6552556201_count_rhs_2D4F13B5CC864BCCB5494D78AE38AB23/part-00000:0+160
2014-08-22 18:04:28,592 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1347302405_0032_m_000000_0' done.
2014-08-22 18:04:28,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1347302405_0032_m_000000_0
2014-08-22 18:04:28,592 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1347302405_0032_m_000001_0
2014-08-22 18:04:28,593 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1c4233fd
2014-08-22 18:04:28,594 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000
2014-08-22 18:04:28,594 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@77383942
2014-08-22 18:04:28,595 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,596 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,727 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,727 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,729 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,729 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,734 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][461280548/source/]
2014-08-22 18:04:28,734 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:28,735 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,737 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,739 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1347302405_0032_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:28,740 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/461280548_source_121BCBF34CEF4E2488FAF3199E85BC59/part-00000:0+160
2014-08-22 18:04:28,741 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1347302405_0032_m_000001_0' done.
2014-08-22 18:04:28,741 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1347302405_0032_m_000001_0
2014-08-22 18:04:28,741 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:28,746 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@628fb58
2014-08-22 18:04:28,747 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,748 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:28,748 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 154 bytes
2014-08-22 18:04:28,749 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,749 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:28,749 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,754 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(source*count-rhs)[by: source:[{1}:'char'] count-rhs:[{1}:'char']]
2014-08-22 18:04:28,754 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5539825644/source_count-rhs/]
2014-08-22 18:04:28,764 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2014-08-22 18:04:28,764 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2014-08-22 18:04:28,772 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1347302405_0032_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,774 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:28,774 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1347302405_0032_r_000000_0 is allowed to commit now
2014-08-22 18:04:28,776 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1347302405_0032_r_000000_0' to file:/tmp/hadoop-teamcity/5539825644_source_count_rhs_33F05A0C663F4716A2293BB37D8AB002
2014-08-22 18:04:28,777 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:28,777 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1347302405_0032_r_000000_0' done.
2014-08-22 18:04:28,784 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] starting step: (5/5) ...m/samesourcemergethreeway
2014-08-22 18:04:28,792 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,793 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:28,814 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] submitted hadoop job: job_local1799521797_0033
2014-08-22 18:04:28,815 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count-lhs*source*count...] tracking url: http://localhost:8080/
2014-08-22 18:04:28,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:28,817 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1799521797_0033_m_000000_0
2014-08-22 18:04:28,818 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7a5ad0f0
2014-08-22 18:04:28,820 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5539825644_source_count_rhs_33F05A0C663F4716A2293BB37D8AB002/part-00000
2014-08-22 18:04:28,820 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2586ebf5
2014-08-22 18:04:28,822 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,822 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:28,953 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:28,954 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:28,956 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:28,956 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:28,963 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'num', 'char2', 'count']]"][5539825644/source_count-rhs/]
2014-08-22 18:04:28,963 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-08-22 18:04:28,965 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:28,967 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:28,968 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1799521797_0033_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:28,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/5539825644_source_count_rhs_33F05A0C663F4716A2293BB37D8AB002/part-00000:0+200
2014-08-22 18:04:28,970 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1799521797_0033_m_000000_0' done.
2014-08-22 18:04:28,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1799521797_0033_m_000000_0
2014-08-22 18:04:28,970 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1799521797_0033_m_000001_0
2014-08-22 18:04:28,971 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7584bf1
2014-08-22 18:04:28,972 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/tmp/hadoop-teamcity/5441208656_count_lhs_22DA6773C85A46AAAFD146FC04030CBC/part-00000
2014-08-22 18:04:28,972 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4dd849ae
2014-08-22 18:04:28,974 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:28,974 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:29,235 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:29,236 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:29,237 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:29,237 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,243 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: TempHfs["SequenceFile[['char', 'count']]"][5441208656/count-lhs/]
2014-08-22 18:04:29,243 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-08-22 18:04:29,245 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:29,247 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:29,248 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1799521797_0033_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:29,250 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/tmp/hadoop-teamcity/5441208656_count_lhs_22DA6773C85A46AAAFD146FC04030CBC/part-00000:0+160
2014-08-22 18:04:29,251 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1799521797_0033_m_000001_0' done.
2014-08-22 18:04:29,251 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1799521797_0033_m_000001_0
2014-08-22 18:04:29,251 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:29,256 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@10942dfb
2014-08-22 18:04:29,257 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,257 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:29,258 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 194 bytes
2014-08-22 18:04:29,258 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,259 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:29,259 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,264 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: CoGroup(count-lhs*source*count-rhs)[by: count-lhs:[{1}:'char'] source*count-rhs:[{1}:'char']]
2014-08-22 18:04:29,265 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['char', 'count']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway"]
2014-08-22 18:04:29,268 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(105)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2014-08-22 18:04:29,268 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(110)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2014-08-22 18:04:29,270 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1799521797_0033_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:29,271 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,271 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1799521797_0033_r_000000_0 is allowed to commit now
2014-08-22 18:04:29,273 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1799521797_0033_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway
2014-08-22 18:04:29,274 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:29,274 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1799521797_0033_r_000000_0' done.
2014-08-22 18:04:29,282 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/samesourcemergethreeway/_temporary
2014-08-22 18:04:29,291 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,300 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,301 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/samesourcemergethreeway
2014-08-22 18:04:29,314 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:29,315 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] starting
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] parallel execution is enabled: false
2014-08-22 18:04:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] starting jobs: 1
2014-08-22 18:04:29,332 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [lhs+rhs] allocating threads: 1
2014-08-22 18:04:29,332 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] starting step: (1/1) ...quemerge-nondeterministic
2014-08-22 18:04:29,340 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,341 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,361 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] submitted hadoop job: job_local2036944319_0034
2014-08-22 18:04:29,361 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [lhs+rhs] tracking url: http://localhost:8080/
2014-08-22 18:04:29,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:29,364 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2036944319_0034_m_000000_0
2014-08-22 18:04:29,365 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@1a4f27a3
2014-08-22 18:04:29,366 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:29,367 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2b995106
2014-08-22 18:04:29,367 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:29,368 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:29,386 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:29,386 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:29,388 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:29,388 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,393 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:29,393 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-08-22 18:04:29,397 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:29,398 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:29,400 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2036944319_0034_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:29,401 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:29,402 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2036944319_0034_m_000000_0' done.
2014-08-22 18:04:29,402 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2036944319_0034_m_000000_0
2014-08-22 18:04:29,402 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local2036944319_0034_m_000001_0
2014-08-22 18:04:29,403 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@29cb6895
2014-08-22 18:04:29,404 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt
2014-08-22 18:04:29,404 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@56277446
2014-08-22 18:04:29,405 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:29,405 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:29,527 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:29,527 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:29,529 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:29,529 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,533 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt"]
2014-08-22 18:04:29,534 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-08-22 18:04:29,536 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:29,538 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:29,539 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2036944319_0034_m_000001_0 is done. And is in the process of commiting
2014-08-22 18:04:29,541 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/rhs.txt:0+51
2014-08-22 18:04:29,541 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2036944319_0034_m_000001_0' done.
2014-08-22 18:04:29,541 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local2036944319_0034_m_000001_0
2014-08-22 18:04:29,541 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:29,548 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@79011694
2014-08-22 18:04:29,548 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,549 INFO mapred.Merger (Merger.java:merge(408)) - Merging 2 sorted segments
2014-08-22 18:04:29,550 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 2 segments left of total size: 174 bytes
2014-08-22 18:04:29,551 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,551 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:29,552 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,557 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(lhs+rhs)[by: lhs:[{1}:'num'] rhs:[{1}:'num']]
2014-08-22 18:04:29,557 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic"]
2014-08-22 18:04:29,562 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local2036944319_0034_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:29,564 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,564 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local2036944319_0034_r_000000_0 is allowed to commit now
2014-08-22 18:04:29,566 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local2036944319_0034_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic
2014-08-22 18:04:29,567 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:29,568 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local2036944319_0034_r_000000_0' done.
2014-08-22 18:04:29,576 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic/_temporary
2014-08-22 18:04:29,584 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,586 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/uniquemerge-nondeterministic
2014-08-22 18:04:29,595 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:29,596 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:29,605 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting
2014-08-22 18:04:29,605 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:29,606 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] sink: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique"]
2014-08-22 18:04:29,606 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] parallel execution is enabled: false
2014-08-22 18:04:29,606 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] starting jobs: 1
2014-08-22 18:04:29,606 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [shape] allocating threads: 1
2014-08-22 18:04:29,606 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] starting step: (1/1) ...blyhelpersplatform/unique
2014-08-22 18:04:29,614 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,635 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] submitted hadoop job: job_local858704912_0035
2014-08-22 18:04:29,636 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [shape] tracking url: http://localhost:8080/
2014-08-22 18:04:29,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:29,638 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local858704912_0035_m_000000_0
2014-08-22 18:04:29,639 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2398724d
2014-08-22 18:04:29,640 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:29,641 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@62513bb5
2014-08-22 18:04:29,642 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:29,642 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:29,779 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:29,779 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:29,780 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:29,780 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,785 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:29,785 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(shape)[by:[{1}:'num']]
2014-08-22 18:04:29,787 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:29,808 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:29,810 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local858704912_0035_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:29,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:29,812 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local858704912_0035_m_000000_0' done.
2014-08-22 18:04:29,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local858704912_0035_m_000000_0
2014-08-22 18:04:29,812 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:29,815 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@22ff0470
2014-08-22 18:04:29,816 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,816 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:29,817 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 87 bytes
2014-08-22 18:04:29,817 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,817 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:29,817 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:29,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(shape)[by:[{1}:'num']]
2014-08-22 18:04:29,821 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['item']->['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique"]
2014-08-22 18:04:29,825 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local858704912_0035_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:29,826 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:29,826 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local858704912_0035_r_000000_0 is allowed to commit now
2014-08-22 18:04:29,828 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local858704912_0035_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique
2014-08-22 18:04:29,829 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:29,829 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local858704912_0035_r_000000_0' done.
2014-08-22 18:04:29,839 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/unique/_temporary
2014-08-22 18:04:29,848 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,850 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/unique
2014-08-22 18:04:29,861 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2014-08-22 18:04:29,861 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.6/a36851b185807587c9a4b9e914f06c5ea27801c9/cascading-hadoop-2.5.6.jar
2014-08-22 18:04:29,872 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting
2014-08-22 18:04:29,872 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] source: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:29,873 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] sink: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull"]
2014-08-22 18:04:29,873 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] parallel execution is enabled: false
2014-08-22 18:04:29,873 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] starting jobs: 1
2014-08-22 18:04:29,873 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [count] allocating threads: 1
2014-08-22 18:04:29,874 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] starting step: (1/1) ...platform/countnullnotnull
2014-08-22 18:04:29,881 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:29,901 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] submitted hadoop job: job_local1979753042_0036
2014-08-22 18:04:29,901 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [count] tracking url: http://localhost:8080/
2014-08-22 18:04:29,903 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(340)) - Waiting for map tasks
2014-08-22 18:04:29,904 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(204)) - Starting task: attempt_local1979753042_0036_m_000000_0
2014-08-22 18:04:29,905 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@3bd37f3b
2014-08-22 18:04:29,906 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt
2014-08-22 18:04:29,906 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(455)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@2bfd96c5
2014-08-22 18:04:29,908 INFO mapred.MapTask (MapTask.java:runOldMapper(419)) - numReduceTasks: 1
2014-08-22 18:04:29,908 INFO mapred.MapTask (MapTask.java:<init>(949)) - io.sort.mb = 100
2014-08-22 18:04:30,044 INFO mapred.MapTask (MapTask.java:<init>(961)) - data buffer = 79691776/99614720
2014-08-22 18:04:30,045 INFO mapred.MapTask (MapTask.java:<init>(962)) - record buffer = 262144/327680
2014-08-22 18:04:30,046 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.5.6
2014-08-22 18:04:30,046 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2014-08-22 18:04:30,053 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextDelimited[['num', 'char']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt"]
2014-08-22 18:04:30,053 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(count)[by:[{?}:NONE]]
2014-08-22 18:04:30,054 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-08-22 18:04:30,060 INFO mapred.MapTask (MapTask.java:flush(1289)) - Starting flush of map output
2014-08-22 18:04:30,061 INFO mapred.MapTask (MapTask.java:sortAndSpill(1471)) - Finished spill 0
2014-08-22 18:04:30,063 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1979753042_0036_m_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:30,065 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/classes/test/data/lhs.txt:0+52
2014-08-22 18:04:30,065 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1979753042_0036_m_000000_0' done.
2014-08-22 18:04:30,066 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(229)) - Finishing task: attempt_local1979753042_0036_m_000000_0
2014-08-22 18:04:30,066 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(348)) - Map task executor complete.
2014-08-22 18:04:30,069 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@7932e6fd
2014-08-22 18:04:30,069 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:30,070 INFO mapred.Merger (Merger.java:merge(408)) - Merging 1 sorted segments
2014-08-22 18:04:30,070 INFO mapred.Merger (Merger.java:merge(491)) - Down to the last merge-pass, with 1 segments left of total size: 10 bytes
2014-08-22 18:04:30,071 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:30,071 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.5.6
2014-08-22 18:04:30,071 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2014-08-22 18:04:30,076 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(count)[by:[{?}:NONE]]
2014-08-22 18:04:30,077 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextDelimited[['notnull', 'null']]"]["/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull"]
2014-08-22 18:04:30,080 INFO mapred.Task (Task.java:done(858)) - Task:attempt_local1979753042_0036_r_000000_0 is done. And is in the process of commiting
2014-08-22 18:04:30,081 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) -
2014-08-22 18:04:30,081 INFO mapred.Task (Task.java:commit(1011)) - Task attempt_local1979753042_0036_r_000000_0 is allowed to commit now
2014-08-22 18:04:30,083 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local1979753042_0036_r_000000_0' to file:/opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull
2014-08-22 18:04:30,084 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(466)) - reduce > reduce
2014-08-22 18:04:30,085 INFO mapred.Task (Task.java:sendDone(970)) - Task 'attempt_local1979753042_0036_r_000000_0' done.
2014-08-22 18:04:30,095 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output/hadoop/assemblyhelpersplatform/countnullnotnull/_temporary
2014-08-22 18:04:30,103 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:30,112 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-08-22 18:04:30,113 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/3945655b04626318/apache-1.2.x/build/test/output//hadoop/assemblyhelpersplatform/countnullnotnull