2015-02-12 16:12:28,216 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-12 16:12:28,246 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-12 16:12:28,247 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.function.FunctionPlatformTest
2015-02-12 16:12:28,259 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-12 16:12:28,739 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-12 16:12:28,863 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 16:12:28,868 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 16:12:28,868 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 223, total: 245, max: 672
2015-02-12 16:12:28,869 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 16:12:28,869 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 16:12:28,870 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 223, total: 245, max: 672
2015-02-12 16:12:28,870 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 36% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 16:12:28,957 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 16:12:28,958 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 16:12:28,968 INFO property.AppProps (AppProps.java:getAppID(169)) - using app.id: DC97E9D8027F4EC09B2DD54CCEF1DF91
2015-02-12 16:12:29,031 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 16:12:29,054 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 16:12:29,056 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 16:12:29,143 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-12 16:12:29,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] starting
2015-02-12 16:12:29,145 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt"]
2015-02-12 16:12:29,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] sink: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/formatter"]
2015-02-12 16:12:29,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] parallel execution is enabled: false
2015-02-12 16:12:29,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] starting jobs: 1
2015-02-12 16:12:29,146 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [formatter] allocating threads: 1
2015-02-12 16:12:29,147 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] starting step: (1/1) ...unctionplatform/formatter
2015-02-12 16:12:29,174 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-12 16:12:29,176 INFO jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-12 16:12:29,196 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 16:12:29,382 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:29,414 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 16:12:29,547 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local1289461259_0001
2015-02-12 16:12:29,570 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1289461259/.staging/job_local1289461259_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:29,579 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi1289461259/.staging/job_local1289461259_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:29,716 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1289461259_0001/job_local1289461259_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:29,722 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local1289461259_0001/job_local1289461259_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:29,726 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 16:12:29,726 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 16:12:29,727 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 16:12:29,729 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] submitted hadoop job: job_local1289461259_0001
2015-02-12 16:12:29,729 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [formatter] tracking url: http://localhost:8080/
2015-02-12 16:12:29,743 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 16:12:29,745 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1289461259_0001_m_000000_0
2015-02-12 16:12:29,770 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 16:12:29,770 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 16:12:29,776 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt
2015-02-12 16:12:29,776 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6a850415
2015-02-12 16:12:29,783 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 16:12:29,796 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 16:12:29,796 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 16:12:29,827 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-12 16:12:29,828 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt"]
2015-02-12 16:12:29,829 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/formatter"]
2015-02-12 16:12:29,839 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 16:12:29,840 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local1289461259_0001_m_000000_0 is done. And is in the process of committing
2015-02-12 16:12:29,845 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 16:12:29,845 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local1289461259_0001_m_000000_0 is allowed to commit now
2015-02-12 16:12:29,846 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1289461259_0001_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/formatter/_temporary/0/task_local1289461259_0001_m_000000
2015-02-12 16:12:29,847 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt:0+19
2015-02-12 16:12:29,847 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1289461259_0001_m_000000_0' done.
2015-02-12 16:12:29,848 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1289461259_0001_m_000000_0
2015-02-12 16:12:29,848 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 16:12:29,880 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/formatter/_temporary
2015-02-12 16:12:29,913 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:29,950 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:29,952 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/functionplatform/formatter
2015-02-12 16:12:29,953 INFO assembly.AggregateBy (AggregateBy.java:prepare(267)) - using threshold value: 2
2015-02-12 16:12:29,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 1, with threshold: 2
2015-02-12 16:12:29,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 268, total: 310, max: 672
2015-02-12 16:12:29,954 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 16:12:29,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(327)) - flushed keys num times: 3, with threshold: 2
2015-02-12 16:12:29,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(328)) - mem on flush (mb), free: 268, total: 310, max: 672
2015-02-12 16:12:29,955 INFO assembly.AggregateBy (AggregateBy.java:removeEldestEntry(333)) - total mem is 46% of max mem, to better utilize unused memory consider increasing current LRU threshold with system property "cascading.aggregateby.threshold"
2015-02-12 16:12:29,981 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 16:12:29,982 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 16:12:29,994 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 16:12:29,999 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 16:12:30,000 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 16:12:30,016 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] starting
2015-02-12 16:12:30,016 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 16:12:30,017 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] sink: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/insert"]
2015-02-12 16:12:30,017 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] parallel execution is enabled: false
2015-02-12 16:12:30,017 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] starting jobs: 1
2015-02-12 16:12:30,017 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [apache] allocating threads: 1
2015-02-12 16:12:30,018 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] starting step: (1/1) ...1/functionplatform/insert
2015-02-12 16:12:30,018 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 16:12:30,020 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 16:12:30,052 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:30,075 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 16:12:30,104 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local300962666_0002
2015-02-12 16:12:30,122 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi300962666/.staging/job_local300962666_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:30,125 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi300962666/.staging/job_local300962666_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:30,190 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local300962666_0002/job_local300962666_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:30,194 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local300962666_0002/job_local300962666_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:30,195 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 16:12:30,195 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 16:12:30,195 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 16:12:30,195 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] submitted hadoop job: job_local300962666_0002
2015-02-12 16:12:30,196 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [apache] tracking url: http://localhost:8080/
2015-02-12 16:12:30,198 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 16:12:30,198 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local300962666_0002_m_000000_0
2015-02-12 16:12:30,200 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 16:12:30,200 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 16:12:30,201 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt
2015-02-12 16:12:30,202 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@5ae31e74
2015-02-12 16:12:30,203 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 1
2015-02-12 16:12:30,271 INFO mapred.MapTask (MapTask.java:setEquator(1202)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-12 16:12:30,271 INFO mapred.MapTask (MapTask.java:init(995)) - mapreduce.task.io.sort.mb: 100
2015-02-12 16:12:30,271 INFO mapred.MapTask (MapTask.java:init(996)) - soft limit at 83886080
2015-02-12 16:12:30,271 INFO mapred.MapTask (MapTask.java:init(997)) - bufstart = 0; bufvoid = 104857600
2015-02-12 16:12:30,272 INFO mapred.MapTask (MapTask.java:init(998)) - kvstart = 26214396; length = 6553600
2015-02-12 16:12:30,282 INFO mapred.MapTask (MapTask.java:createSortingCollector(402)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-12 16:12:30,283 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 16:12:30,283 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 16:12:30,294 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt"]
2015-02-12 16:12:30,294 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(apache)[by:[{1}:'A']]
2015-02-12 16:12:30,310 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 16:12:30,310 INFO mapred.MapTask (MapTask.java:flush(1457)) - Starting flush of map output
2015-02-12 16:12:30,310 INFO mapred.MapTask (MapTask.java:flush(1475)) - Spilling map output
2015-02-12 16:12:30,311 INFO mapred.MapTask (MapTask.java:flush(1476)) - bufstart = 0; bufend = 3000; bufvoid = 104857600
2015-02-12 16:12:30,311 INFO mapred.MapTask (MapTask.java:flush(1478)) - kvstart = 26214396(104857584); kvend = 26213600(104854400); length = 797/6553600
2015-02-12 16:12:30,322 INFO mapred.MapTask (MapTask.java:sortAndSpill(1660)) - Finished spill 0
2015-02-12 16:12:30,326 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local300962666_0002_m_000000_0 is done. And is in the process of committing
2015-02-12 16:12:30,328 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/apache.200.txt:0+32599
2015-02-12 16:12:30,328 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local300962666_0002_m_000000_0' done.
2015-02-12 16:12:30,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local300962666_0002_m_000000_0
2015-02-12 16:12:30,329 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 16:12:30,330 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-12 16:12:30,331 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local300962666_0002_r_000000_0
2015-02-12 16:12:30,335 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 16:12:30,335 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 16:12:30,337 INFO mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@217bd515
2015-02-12 16:12:30,344 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(195)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-12 16:12:30,346 INFO reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local300962666_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-12 16:12:30,369 INFO reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(141)) - localfetcher#1 about to shuffle output of map attempt_local300962666_0002_m_000000_0 decomp: 3402 len: 3406 to MEMORY
2015-02-12 16:12:30,371 INFO reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 3402 bytes from map-output for attempt_local300962666_0002_m_000000_0
2015-02-12 16:12:30,394 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(313)) - closeInMemoryFile -> map-output of size: 3402, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->3402
2015-02-12 16:12:30,394 INFO reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-12 16:12:30,395 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 16:12:30,395 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(673)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-12 16:12:30,400 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 16:12:30,400 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 3393 bytes
2015-02-12 16:12:30,405 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(750)) - Merged 1 segments, 3402 bytes to disk to satisfy reduce memory limit
2015-02-12 16:12:30,405 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(780)) - Merging 1 files, 3406 bytes from disk
2015-02-12 16:12:30,406 INFO reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(795)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-12 16:12:30,406 INFO mapred.Merger (Merger.java:merge(606)) - Merging 1 sorted segments
2015-02-12 16:12:30,407 INFO mapred.Merger (Merger.java:merge(705)) - Down to the last merge-pass, with 1 segments left of total size: 3393 bytes
2015-02-12 16:12:30,407 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 16:12:30,408 INFO hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-12 16:12:30,408 INFO hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-12 16:12:30,419 INFO hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(apache)[by:[{1}:'A']]
2015-02-12 16:12:30,419 INFO hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/insert"]
2015-02-12 16:12:30,455 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local300962666_0002_r_000000_0 is done. And is in the process of committing
2015-02-12 16:12:30,456 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-12 16:12:30,456 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local300962666_0002_r_000000_0 is allowed to commit now
2015-02-12 16:12:30,457 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local300962666_0002_r_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/insert/_temporary/0/task_local300962666_0002_r_000000
2015-02-12 16:12:30,458 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-12 16:12:30,458 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local300962666_0002_r_000000_0' done.
2015-02-12 16:12:30,458 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local300962666_0002_r_000000_0
2015-02-12 16:12:30,458 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-12 16:12:30,476 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/insert/_temporary
2015-02-12 16:12:30,515 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:30,554 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:30,557 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/functionplatform/insert
2015-02-12 16:12:30,591 INFO util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-12 16:12:30,591 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /Users/soberoi/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-12 16:12:30,602 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-12 16:12:30,607 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-12 16:12:30,608 INFO Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1022)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-12 16:12:30,620 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] starting
2015-02-12 16:12:30,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt"]
2015-02-12 16:12:30,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] sink: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/setvalue"]
2015-02-12 16:12:30,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] parallel execution is enabled: false
2015-02-12 16:12:30,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] starting jobs: 1
2015-02-12 16:12:30,621 INFO flow.Flow (BaseFlow.java:logInfo(1378)) - [setvalue] allocating threads: 1
2015-02-12 16:12:30,622 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] starting step: (1/1) ...functionplatform/setvalue
2015-02-12 16:12:30,622 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 16:12:30,624 INFO jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-12 16:12:30,660 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:30,682 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(492)) - number of splits:1
2015-02-12 16:12:30,763 INFO mapreduce.JobSubmitter (JobSubmitter.java:printTokens(575)) - Submitting tokens for job: job_local908516661_0003
2015-02-12 16:12:30,785 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi908516661/.staging/job_local908516661_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:30,786 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/tmp/cascading/staging/soberoi908516661/.staging/job_local908516661_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:30,877 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local908516661_0003/job_local908516661_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-02-12 16:12:30,879 WARN conf.Configuration (Configuration.java:loadProperty(2510)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/tmp/mapred/local/localRunner/soberoi/job_local908516661_0003/job_local908516661_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-02-12 16:12:30,880 INFO mapreduce.Job (Job.java:submit(1299)) - The url to track the job: http://localhost:8080/
2015-02-12 16:12:30,880 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-12 16:12:30,880 INFO mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-12 16:12:30,880 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] submitted hadoop job: job_local908516661_0003
2015-02-12 16:12:30,881 INFO flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [setvalue] tracking url: http://localhost:8080/
2015-02-12 16:12:30,882 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-12 16:12:30,882 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local908516661_0003_m_000000_0
2015-02-12 16:12:30,884 INFO util.ProcfsBasedProcessTree (ProcfsBasedProcessTree.java:isAvailable(181)) - ProcfsBasedProcessTree currently is supported only on Linux.
2015-02-12 16:12:30,884 INFO mapred.Task (Task.java:initialize(587)) - Using ResourceCalculatorProcessTree : null
2015-02-12 16:12:30,886 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt
2015-02-12 16:12:30,886 INFO mapred.MapTask (MapTask.java:updateJobWithSplit(482)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@6d4a1c70
2015-02-12 16:12:30,887 INFO mapred.MapTask (MapTask.java:runOldMapper(436)) - numReduceTasks: 0
2015-02-12 16:12:30,896 INFO hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-12 16:12:30,896 INFO hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-12 16:12:30,903 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt"]
2015-02-12 16:12:30,903 INFO hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['line']->[ALL]]"]["/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/setvalue"]
2015-02-12 16:12:30,906 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 16:12:30,906 INFO mapred.Task (Task.java:done(1001)) - Task:attempt_local908516661_0003_m_000000_0 is done. And is in the process of committing
2015-02-12 16:12:30,907 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) -
2015-02-12 16:12:30,907 INFO mapred.Task (Task.java:commit(1162)) - Task attempt_local908516661_0003_m_000000_0 is allowed to commit now
2015-02-12 16:12:30,908 INFO output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local908516661_0003_m_000000_0' to file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/setvalue/_temporary/0/task_local908516661_0003_m_000000
2015-02-12 16:12:30,909 INFO mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/classes/test/data/upper.txt:0+19
2015-02-12 16:12:30,909 INFO mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local908516661_0003_m_000000_0' done.
2015-02-12 16:12:30,910 INFO mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local908516661_0003_m_000000_0
2015-02-12 16:12:30,910 INFO mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-12 16:12:30,935 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output/hadoop2-mr1/functionplatform/setvalue/_temporary
2015-02-12 16:12:30,965 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:31,000 INFO mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-12 16:12:31,002 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /Users/soberoi/src/compatibility/cascading.compatibility/cloudera/build/test/output//hadoop2-mr1/functionplatform/setvalue