2014-04-09 19:11:01,326 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2014-04-09 19:11:01,344 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop
2014-04-09 19:11:01,344 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.function.FunctionPlatformTest
2014-04-09 19:11:01,354 INFO hadoop.HadoopPlatform (HadoopPlatform.java:setUp(112)) - not using cluster
2014-04-09 19:11:01,490 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-09 19:11:01,492 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-09 19:11:01,492 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 427, total: 480, max: 672
2014-04-09 19:11:01,492 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-09 19:11:01,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-09 19:11:01,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 427, total: 480, max: 672
2014-04-09 19:11:01,493 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-09 19:11:01,531 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-09 19:11:01,532 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.3/28aa68c14ec7f7fde8a1753ed3b6ac8e328e81dc/cascading-hadoop-2.5.3.jar
2014-04-09 19:11:01,538 INFO property.AppProps (AppProps.java:getAppID(162)) - using app.id: 4B038DC496144DF49AAC53B1ADAB2237
2014-04-09 19:11:01,630 INFO util.Version (Version.java:printBanner(78)) - Concurrent, Inc - Cascading 2.5.3
2014-04-09 19:11:01,632 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] starting
2014-04-09 19:11:01,634 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt"]
2014-04-09 19:11:01,635 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] sink: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/formatter"]
2014-04-09 19:11:01,635 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] parallel execution is enabled: false
2014-04-09 19:11:01,635 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] starting jobs: 1
2014-04-09 19:11:01,635 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [formatter] allocating threads: 1
2014-04-09 19:11:01,637 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] starting step: (1/1) ...unctionplatform/formatter
2014-04-09 19:11:01,656 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(52)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2014-04-09 19:11:01,680 WARN snappy.LoadSnappy (LoadSnappy.java:<clinit>(46)) - Snappy native library not loaded
2014-04-09 19:11:01,681 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:01,775 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] submitted hadoop job: job_local_0001
2014-04-09 19:11:01,776 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] tracking url: http://localhost:8080/
2014-04-09 19:11:01,802 INFO util.ProcessTree (ProcessTree.java:isSetsidSupported(63)) - setsid exited with exit code 0
2014-04-09 19:11:01,806 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@4f4e18a1
2014-04-09 19:11:01,813 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt
2014-04-09 19:11:01,819 INFO mapred.MapTask (MapTask.java:runOldMapper(425)) - numReduceTasks: 0
2014-04-09 19:11:01,823 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-09 19:11:01,824 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-09 19:11:01,842 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt"]
2014-04-09 19:11:01,843 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/formatter"]
2014-04-09 19:11:01,848 INFO mapred.Task (Task.java:done(847)) - Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
2014-04-09 19:11:01,849 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) -
2014-04-09 19:11:01,850 INFO mapred.Task (Task.java:commit(1000)) - Task attempt_local_0001_m_000000_0 is allowed to commit now
2014-04-09 19:11:01,852 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local_0001_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/formatter
2014-04-09 19:11:04,795 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt:0+19
2014-04-09 19:11:04,797 INFO mapred.Task (Task.java:sendDone(959)) - Task 'attempt_local_0001_m_000000_0' done.
2014-04-09 19:11:04,807 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/formatter/_temporary
2014-04-09 19:11:04,826 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:04,838 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:04,839 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output//hadoop/functionplatform/formatter
2014-04-09 19:11:04,840 INFO assembly.AggregateBy (AggregateBy.java:prepare(248)) - using threshold value: 2
2014-04-09 19:11:04,840 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 1, with threshold: 2
2014-04-09 19:11:04,840 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 389, total: 480, max: 672
2014-04-09 19:11:04,841 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-09 19:11:04,841 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(307)) - flushed keys num times: 3, with threshold: 2
2014-04-09 19:11:04,841 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(308)) - mem on flush (mb), free: 389, total: 480, max: 672
2014-04-09 19:11:04,841 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(313)) - total mem is 71% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2014-04-09 19:11:04,854 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-09 19:11:04,855 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.3/28aa68c14ec7f7fde8a1753ed3b6ac8e328e81dc/cascading-hadoop-2.5.3.jar
2014-04-09 19:11:04,873 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] starting
2014-04-09 19:11:04,874 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/apache.200.txt"]
2014-04-09 19:11:04,874 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] sink: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/insert"]
2014-04-09 19:11:04,874 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] parallel execution is enabled: false
2014-04-09 19:11:04,874 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] starting jobs: 1
2014-04-09 19:11:04,874 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [apache] allocating threads: 1
2014-04-09 19:11:04,874 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] starting step: (1/1) ...p/functionplatform/insert
2014-04-09 19:11:04,882 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:04,920 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] submitted hadoop job: job_local_0002
2014-04-09 19:11:04,921 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] tracking url: http://localhost:8080/
2014-04-09 19:11:04,923 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@71ffd9f1
2014-04-09 19:11:04,924 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/apache.200.txt
2014-04-09 19:11:04,924 INFO mapred.MapTask (MapTask.java:runOldMapper(425)) - numReduceTasks: 1
2014-04-09 19:11:04,928 INFO mapred.MapTask (MapTask.java:<init>(944)) - io.sort.mb = 100
2014-04-09 19:11:04,938 INFO mapred.MapTask (MapTask.java:<init>(956)) - data buffer = 79691776/99614720
2014-04-09 19:11:04,938 INFO mapred.MapTask (MapTask.java:<init>(957)) - record buffer = 262144/327680
2014-04-09 19:11:04,947 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-09 19:11:04,947 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-09 19:11:04,957 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/apache.200.txt"]
2014-04-09 19:11:04,957 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: GroupBy(apache)[by:[{1}:'A']]
2014-04-09 19:11:04,982 INFO mapred.MapTask (MapTask.java:flush(1284)) - Starting flush of map output
2014-04-09 19:11:04,991 INFO mapred.MapTask (MapTask.java:sortAndSpill(1466)) - Finished spill 0
2014-04-09 19:11:04,993 INFO mapred.Task (Task.java:done(847)) - Task:attempt_local_0002_m_000000_0 is done. And is in the process of commiting
2014-04-09 19:11:07,923 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/apache.200.txt:0+32599
2014-04-09 19:11:07,924 INFO mapred.Task (Task.java:sendDone(959)) - Task 'attempt_local_0002_m_000000_0' done.
2014-04-09 19:11:07,931 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@95660e7
2014-04-09 19:11:07,931 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) -
2014-04-09 19:11:07,934 INFO mapred.Merger (Merger.java:merge(390)) - Merging 1 sorted segments
2014-04-09 19:11:07,935 INFO mapred.Merger (Merger.java:merge(473)) - Down to the last merge-pass, with 1 segments left of total size: 3402 bytes
2014-04-09 19:11:07,935 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) -
2014-04-09 19:11:07,936 INFO hadoop.FlowReducer (FlowReducer.java:configure(77)) - cascading version: 2.5.3
2014-04-09 19:11:07,936 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - child jvm opts: -Xmx200m
2014-04-09 19:11:07,943 INFO hadoop.FlowReducer (FlowReducer.java:configure(96)) - sourcing from: GroupBy(apache)[by:[{1}:'A']]
2014-04-09 19:11:07,943 INFO hadoop.FlowReducer (FlowReducer.java:configure(99)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/insert"]
2014-04-09 19:11:07,959 INFO mapred.Task (Task.java:done(847)) - Task:attempt_local_0002_r_000000_0 is done. And is in the process of commiting
2014-04-09 19:11:07,960 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) -
2014-04-09 19:11:07,960 INFO mapred.Task (Task.java:commit(1000)) - Task attempt_local_0002_r_000000_0 is allowed to commit now
2014-04-09 19:11:07,962 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local_0002_r_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/insert
2014-04-09 19:11:10,931 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) - reduce > reduce
2014-04-09 19:11:10,932 INFO mapred.Task (Task.java:sendDone(959)) - Task 'attempt_local_0002_r_000000_0' done.
2014-04-09 19:11:10,945 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/insert/_temporary
2014-04-09 19:11:10,962 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:10,974 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:10,977 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output//hadoop/functionplatform/insert
2014-04-09 19:11:10,987 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(319)) - using default application jar, may cause class not found exceptions on the cluster
2014-04-09 19:11:10,988 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(205)) - using application jar: /home/brock/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop/2.5.3/28aa68c14ec7f7fde8a1753ed3b6ac8e328e81dc/cascading-hadoop-2.5.3.jar
2014-04-09 19:11:11,002 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] starting
2014-04-09 19:11:11,002 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt"]
2014-04-09 19:11:11,003 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] sink: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/setvalue"]
2014-04-09 19:11:11,003 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] parallel execution is enabled: false
2014-04-09 19:11:11,003 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] starting jobs: 1
2014-04-09 19:11:11,003 INFO flow.Flow (BaseFlow.java:logInfo(1354)) - [setvalue] allocating threads: 1
2014-04-09 19:11:11,004 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] starting step: (1/1) ...functionplatform/setvalue
2014-04-09 19:11:11,011 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:11,040 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] submitted hadoop job: job_local_0003
2014-04-09 19:11:11,053 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] tracking url: http://localhost:8080/
2014-04-09 19:11:11,056 INFO mapred.Task (Task.java:initialize(534)) - Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2929d25c
2014-04-09 19:11:11,057 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt
2014-04-09 19:11:11,058 INFO mapred.MapTask (MapTask.java:runOldMapper(425)) - numReduceTasks: 0
2014-04-09 19:11:11,062 INFO hadoop.FlowMapper (FlowMapper.java:configure(70)) - cascading version: 2.5.3
2014-04-09 19:11:11,062 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - child jvm opts: -Xmx200m
2014-04-09 19:11:11,067 INFO hadoop.FlowMapper (FlowMapper.java:configure(86)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt"]
2014-04-09 19:11:11,067 INFO hadoop.FlowMapper (FlowMapper.java:configure(89)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/setvalue"]
2014-04-09 19:11:11,068 INFO mapred.Task (Task.java:done(847)) - Task:attempt_local_0003_m_000000_0 is done. And is in the process of commiting
2014-04-09 19:11:11,069 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) -
2014-04-09 19:11:11,069 INFO mapred.Task (Task.java:commit(1000)) - Task attempt_local_0003_m_000000_0 is allowed to commit now
2014-04-09 19:11:11,071 INFO mapred.FileOutputCommitter (FileOutputCommitter.java:commitTask(138)) - Saved output of task 'attempt_local_0003_m_000000_0' to file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/setvalue
2014-04-09 19:11:14,056 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(321)) - file:/home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/classes/test/data/upper.txt:0+19
2014-04-09 19:11:14,057 INFO mapred.Task (Task.java:sendDone(959)) - Task 'attempt_local_0003_m_000000_0' done.
2014-04-09 19:11:14,060 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output/hadoop/functionplatform/setvalue/_temporary
2014-04-09 19:11:14,074 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:14,088 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(199)) - Total input paths to process : 1
2014-04-09 19:11:14,089 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /home/brock/workspaces/cascading-compat/cascading.compatibility/apache-1.0.x/build/test/output//hadoop/functionplatform/setvalue