2015-11-23 17:10:07,591 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2015-11-23 17:10:07,607 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.stats.hadoop.HadoopStatsPlatformTest, with platform: hadoop2-mr1
2015-11-23 17:10:07,619 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(136)) - using cluster
2015-11-23 17:10:08,210 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: testClusterID
2015-11-23 17:10:08,800 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(125)) - Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
2015-11-23 17:10:12,638 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 17:10:12,840 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 17:10:13,024 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 17:10:13,254 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 17:10:30,348 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2015-11-23 17:10:30,502 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 17:10:30,503 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 17:10:30,545 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: B2CBDE2EF6ED48B996D77CFFF26EDA6D
2015-11-23 17:10:30,660 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.080
2015-11-23 17:10:30,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 4
2015-11-23 17:10:30,664 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 17:10:30,954 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 17:10:30,954 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 17:10:30,994 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.035
2015-11-23 17:10:30,995 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 4
2015-11-23 17:10:30,995 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 17:10:31,026 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.2
2015-11-23 17:10:31,026 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] starting
2015-11-23 17:10:31,033 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] parallel execution of flows is enabled: true
2015-11-23 17:10:31,033 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] executing total flows: 2
2015-11-23 17:10:31,033 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] allocating management threads: 2
2015-11-23 17:10:31,033 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] starting flow: stats1 test
2015-11-23 17:10:31,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] at least one sink is marked for delete
2015-11-23 17:10:31,034 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] starting flow: stats2 test
2015-11-23 17:10:31,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 17:10:31,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] at least one sink is marked for delete
2015-11-23 17:10:31,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 17:10:31,071 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] starting
2015-11-23 17:10:31,072 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] starting
2015-11-23 17:10:31,072 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 17:10:31,072 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats1"]
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats2"]
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] parallel execution of steps is enabled: true
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] parallel execution of steps is enabled: true
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] executing total steps: 2
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] allocating management threads: 2
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] executing total steps: 2
2015-11-23 17:10:31,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] allocating management threads: 2
2015-11-23 17:10:31,075 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] starting step: (1/2)
2015-11-23 17:10:31,077 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] starting step: (1/2)
2015-11-23 17:10:32,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] submitted hadoop job: job_1448269824845_0002
2015-11-23 17:10:32,039 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] tracking url: http://shihc046-public.cn.ibm.com:41970/proxy/application_1448269824845_0002/
2015-11-23 17:10:32,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] submitted hadoop job: job_1448269824845_0001
2015-11-23 17:10:32,043 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] tracking url: http://shihc046-public.cn.ibm.com:41970/proxy/application_1448269824845_0001/
2015-11-23 17:10:41,192 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0001_01_000005 is : 143
2015-11-23 17:10:41,243 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0001_01_000002 is : 143
2015-11-23 17:10:41,289 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0001_01_000004 is : 143
2015-11-23 17:10:41,311 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0001_01_000003 is : 143
2015-11-23 17:10:46,424 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0001_01_000006 is : 143
2015-11-23 17:10:47,613 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] starting step: (2/2) ...tStatsCounters/flowstats2
2015-11-23 17:10:47,766 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] submitted hadoop job: job_1448269824845_0003
2015-11-23 17:10:47,766 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] tracking url: http://shihc046-public.cn.ibm.com:41970/proxy/application_1448269824845_0003/
2015-11-23 17:11:01,372 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0002_01_000005 is : 143
2015-11-23 17:11:01,398 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0002_01_000002 is : 143
2015-11-23 17:11:01,427 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0002_01_000004 is : 143
2015-11-23 17:11:01,503 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0002_01_000003 is : 143
2015-11-23 17:11:06,672 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0002_01_000006 is : 143
2015-11-23 17:11:07,827 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] starting step: (2/2) ...tStatsCounters/flowstats1
2015-11-23 17:11:07,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] submitted hadoop job: job_1448269824845_0004
2015-11-23 17:11:07,968 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] tracking url: http://shihc046-public.cn.ibm.com:41970/proxy/application_1448269824845_0004/
2015-11-23 17:11:21,285 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0003_01_000002 is : 143
2015-11-23 17:11:26,261 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0003_01_000003 is : 143
2015-11-23 17:11:27,452 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats2/_temporary
2015-11-23 17:11:27,458 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] completed in: 00:56.370, using cpu time: 00:05.580
2015-11-23 17:11:27,459 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] completed flow: stats2 test
2015-11-23 17:11:41,197 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0004_01_000002 is : 143
2015-11-23 17:11:46,328 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269824845_0004_01_000003 is : 143
2015-11-23 17:11:47,518 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats1/_temporary
2015-11-23 17:11:47,525 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] completed in: 01:16.444, using cpu time: 00:05.590
2015-11-23 17:11:47,525 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [stats1 test+stats2 test] completed flow: stats1 test
2015-11-23 17:11:50,873 WARN hdfs.DFSClient (DFSInputStream.java:close(669)) - DFSInputStream has been closed already
2015-11-23 17:11:50,915 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:DAC66] retrieved task reports: 4
2015-11-23 17:11:50,919 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:DAC66] added 4, updated: 0 slices, with duration: 00:00.004, total fetched: 4
2015-11-23 17:11:50,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:DAC66] all 4 children are in finished state, have captured final details: true
2015-11-23 17:11:50,942 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:C375A] retrieved task reports: 1
2015-11-23 17:11:50,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:C375A] added 1, updated: 0 slices, with duration: 00:00.002, total fetched: 1
2015-11-23 17:11:50,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:C375A] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:51,018 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:ADADD] retrieved task reports: 1
2015-11-23 17:11:51,020 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:ADADD] added 1, updated: 0 slices, with duration: 00:00.002, total fetched: 1
2015-11-23 17:11:51,020 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:ADADD] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:51,026 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:CE03A] retrieved task reports: 1
2015-11-23 17:11:51,028 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:CE03A] added 1, updated: 0 slices, with duration: 00:00.002, total fetched: 1
2015-11-23 17:11:51,028 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats1 test] [node:CE03A] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:54,185 WARN hdfs.DFSClient (DFSInputStream.java:close(669)) - DFSInputStream has been closed already
2015-11-23 17:11:54,195 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:B531F] retrieved task reports: 4
2015-11-23 17:11:54,198 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:B531F] added 4, updated: 0 slices, with duration: 00:00.002, total fetched: 4
2015-11-23 17:11:54,198 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:B531F] all 4 children are in finished state, have captured final details: true
2015-11-23 17:11:54,223 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:A4420] retrieved task reports: 1
2015-11-23 17:11:54,224 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:A4420] added 1, updated: 0 slices, with duration: 00:00.001, total fetched: 1
2015-11-23 17:11:54,225 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:A4420] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:57,369 WARN hdfs.DFSClient (DFSInputStream.java:close(669)) - DFSInputStream has been closed already
2015-11-23 17:11:57,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:BE722] retrieved task reports: 1
2015-11-23 17:11:57,374 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:BE722] added 1, updated: 0 slices, with duration: 00:00.002, total fetched: 1
2015-11-23 17:11:57,374 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:BE722] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:57,391 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:FB66B] retrieved task reports: 1
2015-11-23 17:11:57,392 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:FB66B] added 1, updated: 0 slices, with duration: 00:00.001, total fetched: 1
2015-11-23 17:11:57,392 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [stats2 test] [node:FB66B] all 1 children are in finished state, have captured final details: true
2015-11-23 17:11:57,409 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats2
2015-11-23 17:11:57,417 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadoopstatsplatform/testStatsCounters/flowstats1