Class cascading.flow.FlowStrategiesPlatformTest

3

tests

0

failures

3.919s

duration

100%

successful

Tests

Test Duration Result
testFlowStepStrategy 0.915s passed
testSkipStrategiesKeep 0.471s passed
testSkipStrategiesReplace 2.533s passed

Standard output

2015-02-11 00:28:57,421 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(187)) - found 1 test platforms from classpath
2015-02-11 00:28:57,494 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(237)) - installing platform: hadoop2-mr1
2015-02-11 00:28:57,495 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(238)) - running test: cascading.flow.FlowStrategiesPlatformTest
2015-02-11 00:28:57,526 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(105)) - not using cluster
2015-02-11 00:28:58,221 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-02-11 00:28:58,526 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-11 00:28:58,528 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-11 00:28:58,540 INFO  property.AppProps (AppProps.java:getAppID(169)) - using app.id: DA37F9827DED477FB0AF2C2958EF424F
2015-02-11 00:28:58,617 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-11 00:28:58,658 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-11 00:28:58,661 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-11 00:28:58,718 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink is marked for delete
2015-02-11 00:28:58,724 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2015-02-11 00:28:58,726 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink is marked for delete
2015-02-11 00:28:58,726 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2015-02-11 00:28:58,783 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 2.6.3
2015-02-11 00:28:58,787 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] starting
2015-02-11 00:28:58,788 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:28:58,789 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/replace"]
2015-02-11 00:28:58,789 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  parallel execution is enabled: false
2015-02-11 00:28:58,789 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  starting jobs: 1
2015-02-11 00:28:58,789 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  allocating threads: 1
2015-02-11 00:28:58,791 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...trategiesplatform/replace
2015-02-11 00:28:58,819 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - session.id is deprecated. Instead, use dfs.metrics.session-id
2015-02-11 00:28:58,820 INFO  jvm.JvmMetrics (JvmMetrics.java:init(76)) - Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-02-11 00:28:58,845 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-11 00:28:59,276 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:28:59,320 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-11 00:28:59,492 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1698111235_0001
2015-02-11 00:28:59,533 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity1698111235/.staging/job_local1698111235_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:28:59,540 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity1698111235/.staging/job_local1698111235_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:28:59,675 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1698111235_0001/job_local1698111235_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:28:59,679 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1698111235_0001/job_local1698111235_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:28:59,685 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-11 00:28:59,687 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-11 00:28:59,690 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1698111235_0001
2015-02-11 00:28:59,690 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2015-02-11 00:28:59,691 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-11 00:28:59,733 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-11 00:28:59,737 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1698111235_0001_m_000000_0
2015-02-11 00:28:59,781 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-11 00:28:59,794 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt
2015-02-11 00:28:59,794 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@62515263
2015-02-11 00:28:59,804 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-11 00:28:59,824 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-11 00:28:59,824 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-11 00:28:59,869 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.task.partition is deprecated. Instead, use mapreduce.task.partition
2015-02-11 00:28:59,891 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:28:59,892 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/replace"]
2015-02-11 00:28:59,911 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-11 00:28:59,911 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1698111235_0001_m_000000_0 is done. And is in the process of committing
2015-02-11 00:28:59,921 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-11 00:28:59,921 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1698111235_0001_m_000000_0 is allowed to commit now
2015-02-11 00:28:59,923 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1698111235_0001_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/replace/_temporary/0/task_local1698111235_0001_m_000000
2015-02-11 00:28:59,925 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt:0+1449
2015-02-11 00:28:59,925 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1698111235_0001_m_000000_0' done.
2015-02-11 00:28:59,925 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1698111235_0001_m_000000_0
2015-02-11 00:28:59,926 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-11 00:28:59,971 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/replace/_temporary
2015-02-11 00:28:59,973 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink is marked for delete
2015-02-11 00:28:59,973 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2015-02-11 00:28:59,974 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink is marked for delete
2015-02-11 00:28:59,975 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Dec 31 23:59:59 UTC 1969
2015-02-11 00:29:00,005 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,051 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,056 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output//hadoop2-mr1/flowstrategiesplatform/replace
2015-02-11 00:29:00,095 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-11 00:29:00,096 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-11 00:29:00,111 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-11 00:29:00,117 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-11 00:29:00,118 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-11 00:29:00,128 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink does not exist
2015-02-11 00:29:00,131 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] at least one sink does not exist
2015-02-11 00:29:00,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] starting
2015-02-11 00:29:00,132 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:29:00,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/keep"]
2015-02-11 00:29:00,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  parallel execution is enabled: false
2015-02-11 00:29:00,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  starting jobs: 1
2015-02-11 00:29:00,133 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  allocating threads: 1
2015-02-11 00:29:00,134 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...owstrategiesplatform/keep
2015-02-11 00:29:00,135 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-11 00:29:00,136 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-11 00:29:00,172 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,198 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-11 00:29:00,236 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local1180201524_0002
2015-02-11 00:29:00,260 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity1180201524/.staging/job_local1180201524_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:29:00,264 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity1180201524/.staging/job_local1180201524_0002/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:29:00,355 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1180201524_0002/job_local1180201524_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:29:00,358 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local1180201524_0002/job_local1180201524_0002.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:29:00,359 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-11 00:29:00,359 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local1180201524_0002
2015-02-11 00:29:00,359 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-11 00:29:00,359 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2015-02-11 00:29:00,360 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-11 00:29:00,365 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-11 00:29:00,365 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local1180201524_0002_m_000000_0
2015-02-11 00:29:00,367 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-11 00:29:00,370 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt
2015-02-11 00:29:00,370 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@625db9eb
2015-02-11 00:29:00,372 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 0
2015-02-11 00:29:00,385 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-11 00:29:00,385 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-11 00:29:00,398 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:29:00,398 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/keep"]
2015-02-11 00:29:00,403 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-11 00:29:00,403 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local1180201524_0002_m_000000_0 is done. And is in the process of committing
2015-02-11 00:29:00,405 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-11 00:29:00,406 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local1180201524_0002_m_000000_0 is allowed to commit now
2015-02-11 00:29:00,407 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local1180201524_0002_m_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/keep/_temporary/0/task_local1180201524_0002_m_000000
2015-02-11 00:29:00,408 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt:0+1449
2015-02-11 00:29:00,409 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local1180201524_0002_m_000000_0' done.
2015-02-11 00:29:00,409 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local1180201524_0002_m_000000_0
2015-02-11 00:29:00,409 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-11 00:29:00,438 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/keep/_temporary
2015-02-11 00:29:00,444 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Feb 11 00:29:00 UTC 2015
2015-02-11 00:29:00,448 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] source modification date at: Tue Feb 10 23:02:23 UTC 2015
2015-02-11 00:29:00,452 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] sink oldest modified date: Wed Feb 11 00:29:00 UTC 2015
2015-02-11 00:29:00,480 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,527 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,530 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output//hadoop2-mr1/flowstrategiesplatform/keep
2015-02-11 00:29:00,570 INFO  util.HadoopUtil (HadoopUtil.java:findMainClass(336)) - using default application jar, may cause class not found exceptions on the cluster
2015-02-11 00:29:00,570 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(225)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/2.6.3/9ae016e2b8d246b9c1de9168e715d9f1b971aa4a/cascading-hadoop2-mr1-2.6.3.jar
2015-02-11 00:29:00,603 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
2015-02-11 00:29:00,608 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2015-02-11 00:29:00,609 INFO  Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1019)) - mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2015-02-11 00:29:00,633 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test] starting
2015-02-11 00:29:00,634 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:29:00,635 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/simple"]
2015-02-11 00:29:00,636 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  parallel execution is enabled: false
2015-02-11 00:29:00,636 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  starting jobs: 1
2015-02-11 00:29:00,637 INFO  flow.Flow (BaseFlow.java:logInfo(1378)) - [test]  allocating threads: 1
2015-02-11 00:29:00,638 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] starting step: (1/1) ...strategiesplatform/simple
2015-02-11 00:29:00,639 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-11 00:29:00,640 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71)) - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2015-02-11 00:29:00,680 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:00,709 INFO  mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(396)) - number of splits:1
2015-02-11 00:29:00,790 INFO  mapreduce.JobSubmitter (JobSubmitter.java:printTokens(479)) - Submitting tokens for job: job_local2010455715_0003
2015-02-11 00:29:00,819 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity2010455715/.staging/job_local2010455715_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:29:00,823 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/tmp/cascading/staging/teamcity2010455715/.staging/job_local2010455715_0003/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:29:00,899 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2010455715_0003/job_local2010455715_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
2015-02-11 00:29:00,901 WARN  conf.Configuration (Configuration.java:loadProperty(2368)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/tmp/mapred/local/localRunner/teamcity/job_local2010455715_0003/job_local2010455715_0003.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
2015-02-11 00:29:00,902 INFO  mapreduce.Job (Job.java:submit(1289)) - The url to track the job: http://localhost:8080/
2015-02-11 00:29:00,903 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] submitted hadoop job: job_local2010455715_0003
2015-02-11 00:29:00,903 INFO  flow.FlowStep (BaseFlowStep.java:logInfo(834)) - [test] tracking url: http://localhost:8080/
2015-02-11 00:29:00,903 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(471)) - OutputCommitter set in config null
2015-02-11 00:29:00,904 INFO  mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(489)) - OutputCommitter is org.apache.hadoop.mapred.FileOutputCommitter
2015-02-11 00:29:00,909 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for map tasks
2015-02-11 00:29:00,909 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(224)) - Starting task: attempt_local2010455715_0003_m_000000_0
2015-02-11 00:29:00,913 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-11 00:29:00,916 INFO  io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt
2015-02-11 00:29:00,916 INFO  mapred.MapTask (MapTask.java:updateJobWithSplit(462)) - Processing split: cascading.tap.hadoop.io.MultiInputSplit@4beb2473
2015-02-11 00:29:00,919 INFO  mapred.MapTask (MapTask.java:runOldMapper(416)) - numReduceTasks: 1
2015-02-11 00:29:00,926 INFO  mapred.MapTask (MapTask.java:createSortingCollector(388)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2015-02-11 00:29:01,094 INFO  mapred.MapTask (MapTask.java:setEquator(1182)) - (EQUATOR) 0 kvi 26214396(104857584)
2015-02-11 00:29:01,095 INFO  mapred.MapTask (MapTask.java:init(975)) - mapreduce.task.io.sort.mb: 100
2015-02-11 00:29:01,095 INFO  mapred.MapTask (MapTask.java:init(976)) - soft limit at 83886080
2015-02-11 00:29:01,095 INFO  mapred.MapTask (MapTask.java:init(977)) - bufstart = 0; bufvoid = 104857600
2015-02-11 00:29:01,095 INFO  mapred.MapTask (MapTask.java:init(978)) - kvstart = 26214396; length = 6553600
2015-02-11 00:29:01,119 INFO  hadoop.FlowMapper (FlowMapper.java:configure(71)) - cascading version: 2.6.3
2015-02-11 00:29:01,119 INFO  hadoop.FlowMapper (FlowMapper.java:configure(72)) - child jvm opts: -Xmx200m
2015-02-11 00:29:01,138 INFO  hadoop.FlowMapper (FlowMapper.java:configure(87)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt"]
2015-02-11 00:29:01,138 INFO  hadoop.FlowMapper (FlowMapper.java:configure(90)) - sinking to: GroupBy(test)[by:[{1}:'ip']]
2015-02-11 00:29:01,143 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 
2015-02-11 00:29:01,143 INFO  mapred.MapTask (MapTask.java:flush(1437)) - Starting flush of map output
2015-02-11 00:29:01,143 INFO  mapred.MapTask (MapTask.java:flush(1455)) - Spilling map output
2015-02-11 00:29:01,143 INFO  mapred.MapTask (MapTask.java:flush(1456)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2015-02-11 00:29:01,144 INFO  mapred.MapTask (MapTask.java:flush(1458)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2015-02-11 00:29:01,173 INFO  mapred.MapTask (MapTask.java:sortAndSpill(1641)) - Finished spill 0
2015-02-11 00:29:01,178 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local2010455715_0003_m_000000_0 is done. And is in the process of committing
2015-02-11 00:29:01,182 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/classes/test/data/apache.10.txt:0+1449
2015-02-11 00:29:01,182 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2010455715_0003_m_000000_0' done.
2015-02-11 00:29:01,183 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(249)) - Finishing task: attempt_local2010455715_0003_m_000000_0
2015-02-11 00:29:01,184 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - map task executor complete.
2015-02-11 00:29:01,186 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(448)) - Waiting for reduce tasks
2015-02-11 00:29:01,187 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(302)) - Starting task: attempt_local2010455715_0003_r_000000_0
2015-02-11 00:29:01,194 INFO  mapred.Task (Task.java:initialize(587)) -  Using ResourceCalculatorProcessTree : [ ]
2015-02-11 00:29:01,197 INFO  mapred.ReduceTask (ReduceTask.java:run(362)) - Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@1ee99d0f
2015-02-11 00:29:01,211 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:<init>(193)) - MergerManager: memoryLimit=493250144, maxSingleShuffleLimit=123312536, mergeThreshold=325545120, ioSortFactor=10, memToMemMergeOutputsThreshold=10
2015-02-11 00:29:01,215 INFO  reduce.EventFetcher (EventFetcher.java:run(61)) - attempt_local2010455715_0003_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
2015-02-11 00:29:01,256 INFO  reduce.LocalFetcher (LocalFetcher.java:copyMapOutput(140)) - localfetcher#1 about to shuffle output of map attempt_local2010455715_0003_m_000000_0 decomp: 232 len: 236 to MEMORY
2015-02-11 00:29:01,259 INFO  reduce.InMemoryMapOutput (InMemoryMapOutput.java:shuffle(100)) - Read 232 bytes from map-output for attempt_local2010455715_0003_m_000000_0
2015-02-11 00:29:01,303 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:closeInMemoryFile(307)) - closeInMemoryFile -> map-output of size: 232, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->232
2015-02-11 00:29:01,305 INFO  reduce.EventFetcher (EventFetcher.java:run(76)) - EventFetcher is interrupted.. Returning
2015-02-11 00:29:01,307 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-11 00:29:01,308 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(667)) - finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs
2015-02-11 00:29:01,318 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-11 00:29:01,318 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2015-02-11 00:29:01,321 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(742)) - Merged 1 segments, 232 bytes to disk to satisfy reduce memory limit
2015-02-11 00:29:01,321 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(772)) - Merging 1 files, 236 bytes from disk
2015-02-11 00:29:01,322 INFO  reduce.MergeManagerImpl (MergeManagerImpl.java:finalMerge(787)) - Merging 0 segments, 0 bytes from memory into reduce
2015-02-11 00:29:01,322 INFO  mapred.Merger (Merger.java:merge(591)) - Merging 1 sorted segments
2015-02-11 00:29:01,323 INFO  mapred.Merger (Merger.java:merge(690)) - Down to the last merge-pass, with 1 segments left of total size: 211 bytes
2015-02-11 00:29:01,324 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-11 00:29:01,325 INFO  hadoop.FlowReducer (FlowReducer.java:configure(78)) - cascading version: 2.6.3
2015-02-11 00:29:01,326 INFO  hadoop.FlowReducer (FlowReducer.java:configure(79)) - child jvm opts: -Xmx200m
2015-02-11 00:29:01,341 INFO  hadoop.FlowReducer (FlowReducer.java:configure(97)) - sourcing from: GroupBy(test)[by:[{1}:'ip']]
2015-02-11 00:29:01,341 INFO  hadoop.FlowReducer (FlowReducer.java:configure(100)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/simple"]
2015-02-11 00:29:01,366 INFO  mapred.Task (Task.java:done(1001)) - Task:attempt_local2010455715_0003_r_000000_0 is done. And is in the process of committing
2015-02-11 00:29:01,368 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - 1 / 1 copied.
2015-02-11 00:29:01,368 INFO  mapred.Task (Task.java:commit(1162)) - Task attempt_local2010455715_0003_r_000000_0 is allowed to commit now
2015-02-11 00:29:01,369 INFO  output.FileOutputCommitter (FileOutputCommitter.java:commitTask(439)) - Saved output of task 'attempt_local2010455715_0003_r_000000_0' to file:/opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/simple/_temporary/0/task_local2010455715_0003_r_000000
2015-02-11 00:29:01,371 INFO  mapred.LocalJobRunner (LocalJobRunner.java:statusUpdate(591)) - reduce > reduce
2015-02-11 00:29:01,371 INFO  mapred.Task (Task.java:sendDone(1121)) - Task 'attempt_local2010455715_0003_r_000000_0' done.
2015-02-11 00:29:01,372 INFO  mapred.LocalJobRunner (LocalJobRunner.java:run(325)) - Finishing task: attempt_local2010455715_0003_r_000000_0
2015-02-11 00:29:01,372 INFO  mapred.LocalJobRunner (LocalJobRunner.java:runTasks(456)) - reduce task executor complete.
2015-02-11 00:29:01,401 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(219)) - deleting temp path /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output/hadoop2-mr1/flowstrategiesplatform/simple/_temporary
2015-02-11 00:29:01,443 INFO  mapred.FileInputFormat (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
2015-02-11 00:29:01,446 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(209)) - copying to local /opt/buildAgent-8.0.3/work/84ab5c9b70b89230/apache-2.5.x/build/test/output//hadoop2-mr1/flowstrategiesplatform/simple