2015-11-23 16:56:38,718 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2015-11-23 16:56:38,729 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.tap.hadoop.HadoopTapPlatformTest, with platform: hadoop2-mr1
2015-11-23 16:56:38,742 INFO hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(136)) - using cluster
2015-11-23 16:56:39,327 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: testClusterID
2015-11-23 16:56:39,904 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(125)) - Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
2015-11-23 16:56:43,859 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:56:44,071 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:56:44,322 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:56:44,512 WARN containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:57:01,625 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2015-11-23 16:57:03,009 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:03,010 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:03,052 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: C498A110FABE450384E35033630CF7E6
2015-11-23 16:57:03,128 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.047
2015-11-23 16:57:03,130 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:57:03,131 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:03,384 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.2
2015-11-23 16:57:03,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:57:03,387 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["CommentScheme[['line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/comments+lower.txt"]
2015-11-23 16:57:03,387 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNullsFromScheme/testnulls"]
2015-11-23 16:57:03,388 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:57:03,389 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:57:03,389 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:57:03,390 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...NullsFromScheme/testnulls
2015-11-23 16:57:04,225 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted hadoop job: job_1448269016123_0001
2015-11-23 16:57:04,226 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0001/
2015-11-23 16:57:12,646 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0001_01_000002 is : 143
2015-11-23 16:57:13,965 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNullsFromScheme/testnulls/_temporary
2015-11-23 16:57:13,971 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:10.576, using cpu time: 00:00.990
2015-11-23 16:57:14,047 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testNullsFromScheme/testnulls
2015-11-23 16:57:14,097 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:14,097 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:14,132 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.029
2015-11-23 16:57:14,133 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:57:14,133 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:14,136 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testMissingInputFormat/missinginputformat
2015-11-23 16:57:14,269 INFO hadoop.Hfs (Hfs.java:makeLocal(428)) - forcing job to local mode, via source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:57:14,314 INFO hadoop.Hfs (Hfs.java:makeLocal(428)) - forcing job to local mode, via source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:57:14,358 INFO hadoop.Hfs (Hfs.java:makeLocal(428)) - forcing job to local mode, via source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:57:14,384 INFO hadoop.Hfs (Hfs.java:makeLocal(428)) - forcing job to local mode, via source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:57:14,406 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:14,406 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:14,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.015
2015-11-23 16:57:14,427 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:57:14,427 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:14,442 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:57:14,443 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[['line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:57:14,443 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["ResolvedScheme[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testResolvedSinkFields/resolvedfields"]
2015-11-23 16:57:14,444 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:57:14,444 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:57:14,444 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:57:14,445 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...SinkFields/resolvedfields
2015-11-23 16:57:14,613 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted hadoop job: job_1448269016123_0002
2015-11-23 16:57:14,613 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0002/
2015-11-23 16:57:28,463 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0002_01_000004 is : 143
2015-11-23 16:57:28,496 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0002_01_000005 is : 143
2015-11-23 16:57:28,533 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0002_01_000003 is : 143
2015-11-23 16:57:28,590 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0002_01_000006 is : 143
2015-11-23 16:57:29,777 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testResolvedSinkFields/resolvedfields/_temporary
2015-11-23 16:57:29,782 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:15.333, using cpu time: 00:04.380
2015-11-23 16:57:29,835 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testResolvedSinkFields/resolvedfields
2015-11-23 16:57:29,890 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:29,890 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:29,910 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.015
2015-11-23 16:57:29,910 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:57:29,910 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:29,913 INFO hadoop.Hfs (Hfs.java:makeLocal(428)) - forcing job to local mode, via source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:57:29,933 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:29,934 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:29,952 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.014
2015-11-23 16:57:29,953 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:57:29,953 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:29,985 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting
2015-11-23 16:57:29,989 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] parallel execution of flows is enabled: true
2015-11-23 16:57:29,989 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] executing total flows: 2
2015-11-23 16:57:29,989 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] allocating management threads: 2
2015-11-23 16:57:29,990 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting flow: first
2015-11-23 16:57:29,991 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] at least one sink is marked for delete
2015-11-23 16:57:29,991 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] starting
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] source: GlobHfs[/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/?{ppe[_r],owe?}.txt]
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob"]
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] parallel execution of steps is enabled: true
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] executing total steps: 1
2015-11-23 16:57:29,998 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] allocating management threads: 1
2015-11-23 16:57:30,000 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] starting step: (1/1) ...platform/testGlobHfs/glob
2015-11-23 16:57:30,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] submitted hadoop job: job_local969971677_0001
2015-11-23 16:57:30,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] tracking url: http://localhost:8080/
2015-11-23 16:57:30,368 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt
2015-11-23 16:57:30,378 INFO hadoop.FlowMapper (FlowMapper.java:configure(75)) - cascading version: 3.0.2
2015-11-23 16:57:30,378 INFO hadoop.FlowMapper (FlowMapper.java:configure(76)) - child jvm opts: -Xmx512m
2015-11-23 16:57:30,390 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, ordinal: 0
2015-11-23 16:57:30,408 INFO hadoop.FlowMapper (FlowMapper.java:configure(94)) - sourcing from: GlobHfs[/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/?{ppe[_r],owe?}.txt]
2015-11-23 16:57:30,408 INFO hadoop.FlowMapper (FlowMapper.java:configure(97)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob"]
2015-11-23 16:57:30,408 INFO hadoop.FlowMapper (LogUtil.java:logMemory(59)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, mem on start (mb), free: 532, total: 616, max: 672
2015-11-23 16:57:30,413 INFO hadoop.FlowMapper (LogUtil.java:logMemory(59)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, mem on close (mb), free: 531, total: 616, max: 672
2015-11-23 16:57:30,439 INFO io.MultiInputSplit (MultiInputSplit.java:readFields(161)) - current split input path: file:/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt
2015-11-23 16:57:30,443 INFO hadoop.FlowMapper (FlowMapper.java:configure(75)) - cascading version: 3.0.2
2015-11-23 16:57:30,443 INFO hadoop.FlowMapper (FlowMapper.java:configure(76)) - child jvm opts: -Xmx512m
2015-11-23 16:57:30,449 INFO hadoop.FlowMapper (FlowMapper.java:configure(87)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, ordinal: 0
2015-11-23 16:57:30,450 INFO hadoop.FlowMapper (FlowMapper.java:configure(94)) - sourcing from: GlobHfs[/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/?{ppe[_r],owe?}.txt]
2015-11-23 16:57:30,450 INFO hadoop.FlowMapper (FlowMapper.java:configure(97)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob"]
2015-11-23 16:57:30,450 INFO hadoop.FlowMapper (LogUtil.java:logMemory(59)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, mem on start (mb), free: 529, total: 616, max: 672
2015-11-23 16:57:30,451 INFO hadoop.FlowMapper (LogUtil.java:logMemory(59)) - flow node id: FD6CF2B9692B49EBB216C0C7D7BFFE65, mem on close (mb), free: 529, total: 616, max: 672
2015-11-23 16:57:30,540 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob/_temporary
2015-11-23 16:57:30,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] completed in: 00:00.541
2015-11-23 16:57:30,542 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] completed flow: first
2015-11-23 16:57:30,542 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting flow: second
2015-11-23 16:57:30,542 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] at least one sink is marked for delete
2015-11-23 16:57:30,543 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 16:57:30,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] starting
2015-11-23 16:57:30,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob"]
2015-11-23 16:57:30,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob2"]
2015-11-23 16:57:30,546 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] parallel execution of steps is enabled: true
2015-11-23 16:57:30,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] executing total steps: 1
2015-11-23 16:57:30,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] allocating management threads: 1
2015-11-23 16:57:30,548 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] starting step: (1/1) ...latform/testGlobHfs/glob2
2015-11-23 16:57:30,904 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] submitted hadoop job: job_1448269016123_0003
2015-11-23 16:57:30,904 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0003/
2015-11-23 16:57:43,187 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0003_01_000002 is : 143
2015-11-23 16:57:43,441 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0003_01_000005 is : 143
2015-11-23 16:57:43,558 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0003_01_000004 is : 143
2015-11-23 16:57:44,187 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0003_01_000003 is : 143
2015-11-23 16:57:45,374 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testGlobHfs/glob2/_temporary
2015-11-23 16:57:45,378 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] completed in: 00:14.824, using cpu time: 00:03.280
2015-11-23 16:57:45,378 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] completed flow: second
2015-11-23 16:57:45,405 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testGlobHfs/glob
2015-11-23 16:57:45,413 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testGlobHfs/glob2
2015-11-23 16:57:45,437 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:57:45,437 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:57:45,458 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.017
2015-11-23 16:57:45,458 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:57:45,459 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:57:45,482 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:57:45,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: ["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:57:45,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: ["TextDelimited[['ip', 'count']]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testPrepareResource/preparetap"]
2015-11-23 16:57:45,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:57:45,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:57:45,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:57:45,485 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...repareResource/preparetap
2015-11-23 16:57:45,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted hadoop job: job_1448269016123_0004
2015-11-23 16:57:45,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0004/
2015-11-23 16:57:59,148 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0004_01_000002 is : 143
2015-11-23 16:57:59,261 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0004_01_000003 is : 143
2015-11-23 16:57:59,496 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0004_01_000005 is : 143
2015-11-23 16:57:59,538 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0004_01_000004 is : 143
2015-11-23 16:58:03,655 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0004_01_000006 is : 143
2015-11-23 16:58:04,840 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testPrepareResource/preparetap/_temporary
2015-11-23 16:58:04,845 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:19.354, using cpu time: 00:04.400
2015-11-23 16:58:04,870 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testPrepareResource/preparetap
2015-11-23 16:58:04,892 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:04,892 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:04,916 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.019
2015-11-23 16:58:04,916 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:58:04,917 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:04,934 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2015-11-23 16:58:04,935 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:58:04,935 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["DupeConfigScheme[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:58:04,935 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[['line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testDupeConfigFromScheme/dupeconfig"]
2015-11-23 16:58:04,935 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: true
2015-11-23 16:58:04,935 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2015-11-23 16:58:04,936 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2015-11-23 16:58:04,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...nfigFromScheme/dupeconfig
2015-11-23 16:58:05,069 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted hadoop job: job_1448269016123_0005
2015-11-23 16:58:05,069 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0005/
2015-11-23 16:58:19,490 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0005_01_000006 is : 143
2015-11-23 16:58:19,970 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0005_01_000004 is : 143
2015-11-23 16:58:19,994 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0005_01_000002 is : 143
2015-11-23 16:58:20,073 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0005_01_000003 is : 143
2015-11-23 16:58:21,031 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0005_01_000005 is : 143
2015-11-23 16:58:22,209 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testDupeConfigFromScheme/dupeconfig/_temporary
2015-11-23 16:58:22,213 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:17.275, using cpu time: 00:04.820
2015-11-23 16:58:22,271 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testDupeConfigFromScheme/dupeconfig
2015-11-23 16:58:22,301 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:22,301 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:22,316 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.011
2015-11-23 16:58:22,317 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:58:22,317 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:22,342 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:22,342 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:22,359 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.013
2015-11-23 16:58:22,360 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:58:22,360 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:22,399 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting
2015-11-23 16:58:22,402 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] parallel execution of flows is enabled: true
2015-11-23 16:58:22,402 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] executing total flows: 2
2015-11-23 16:58:22,402 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] allocating management threads: 2
2015-11-23 16:58:22,402 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting flow: first
2015-11-23 16:58:22,402 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] at least one sink is marked for delete
2015-11-23 16:58:22,403 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 16:58:22,406 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] starting
2015-11-23 16:58:22,406 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] source: MultiSourceTap[2:[GlobHfs[/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/?{ppe[_r]}.txt], GlobHfs[/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/?{owe?}.txt]]]
2015-11-23 16:58:22,407 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultisource"]
2015-11-23 16:58:22,407 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] parallel execution of steps is enabled: true
2015-11-23 16:58:22,407 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] executing total steps: 1
2015-11-23 16:58:22,407 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] allocating management threads: 1
2015-11-23 16:58:22,408 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] starting step: (1/1) ...ceGlobHfs/globmultisource
2015-11-23 16:58:22,464 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:58:22,465 WARN protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:58:22,465 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable: unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:58:22,533 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] submitted hadoop job: job_1448269016123_0006
2015-11-23 16:58:22,534 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0006/
2015-11-23 16:58:22,709 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:58:22,712 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:58:22,712 WARN protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:58:22,712 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable: unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:58:22,712 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:58:22,713 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:58:22,713 WARN protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:58:22,713 WARN blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable: unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:58:37,457 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000006 is : 143
2015-11-23 16:58:37,579 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000002 is : 143
2015-11-23 16:58:37,674 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000003 is : 143
2015-11-23 16:58:37,698 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000005 is : 143
2015-11-23 16:58:37,721 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000004 is : 143
2015-11-23 16:58:38,590 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0006_01_000007 is : 143
2015-11-23 16:58:39,770 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultisource/_temporary
2015-11-23 16:58:39,775 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [first] completed in: 00:17.363, using cpu time: 00:04.750
2015-11-23 16:58:39,776 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] completed flow: first
2015-11-23 16:58:39,776 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] starting flow: second
2015-11-23 16:58:39,776 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] at least one sink is marked for delete
2015-11-23 16:58:39,776 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] sink oldest modified date: Thu Jan 01 07:59:59 CST 1970
2015-11-23 16:58:39,780 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] starting
2015-11-23 16:58:39,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultisource"]
2015-11-23 16:58:39,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultiource2"]
2015-11-23 16:58:39,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] parallel execution of steps is enabled: true
2015-11-23 16:58:39,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] executing total steps: 1
2015-11-23 16:58:39,781 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] allocating management threads: 1
2015-11-23 16:58:39,783 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] starting step: (1/1) ...ceGlobHfs/globmultiource2
2015-11-23 16:58:40,162 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] submitted hadoop job: job_1448269016123_0007
2015-11-23 16:58:40,162 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0007/
2015-11-23 16:58:56,778 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000002 is : 143
2015-11-23 16:58:56,847 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000007 is : 143
2015-11-23 16:58:56,877 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000008 is : 143
2015-11-23 16:58:56,948 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000004 is : 143
2015-11-23 16:58:57,027 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000003 is : 143
2015-11-23 16:58:57,089 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0007_01_000005 is : 143
2015-11-23 16:58:58,269 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultiource2/_temporary
2015-11-23 16:58:58,274 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [second] completed in: 00:18.487, using cpu time: 00:05.760
2015-11-23 16:58:58,274 INFO cascade.Cascade (BaseCascade.java:logInfo(860)) - [first+second] completed flow: second
2015-11-23 16:58:58,303 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultiource2
2015-11-23 16:58:58,317 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testNestedMultiSourceGlobHfs/globmultisource
2015-11-23 16:58:58,388 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:58,389 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:58,412 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.018
2015-11-23 16:58:58,412 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:58:58,413 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: ["TextDelimited[['ip', 'count']]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testPrepareResourceForWriteFails/preparewritetapfail"]
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:58:58,426 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:58:58,429 ERROR flow.Flow (BaseFlow.java:logError(1469)) - [test] unable to prepare tap for write: hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testPrepareResourceForWriteFails/preparewritetapfail
java.io.IOException: failed intentionally
at cascading.tap.hadoop.HadoopTapPlatformTest$7.prepareResourceForWrite(HadoopTapPlatformTest.java:774)
at cascading.tap.hadoop.HadoopTapPlatformTest$7.prepareResourceForWrite(HadoopTapPlatformTest.java:770)
at cascading.flow.planner.BaseFlowStep.prepareResource(BaseFlowStep.java:567)
at cascading.flow.planner.BaseFlowStep.prepareResources(BaseFlowStep.java:549)
at cascading.flow.planner.BaseFlowStep.prepareResources(BaseFlowStep.java:535)
at cascading.flow.planner.FlowStepJob.prepareResources(FlowStepJob.java:217)
at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:180)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:146)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:48)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2015-11-23 16:58:58,436 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping all jobs
2015-11-23 16:58:58,437 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping: (1/1) ...Fails/preparewritetapfail
2015-11-23 16:58:58,438 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopped all jobs
2015-11-23 16:58:58,440 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.012
2015-11-23 16:58:58,440 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testPrepareResourceForWriteFails/preparewritetapfail
2015-11-23 16:58:58,458 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:58,459 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:58,475 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.013
2015-11-23 16:58:58,476 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:58:58,476 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:58,487 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:58:58,487 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: ["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:58:58,487 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextDelimited[['ip', 'count']]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testPrepareResourceForReadFails/preparereadtapfail"]
2015-11-23 16:58:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:58:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:58:58,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:58:58,489 ERROR flow.Flow (BaseFlow.java:logError(1469)) - [test] unable to prepare tap for read: hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt
java.io.IOException: failed intentionally
at cascading.tap.hadoop.HadoopTapPlatformTest$6.prepareResourceForRead(HadoopTapPlatformTest.java:726)
at cascading.tap.hadoop.HadoopTapPlatformTest$6.prepareResourceForRead(HadoopTapPlatformTest.java:722)
at cascading.flow.planner.BaseFlowStep.prepareResource(BaseFlowStep.java:569)
at cascading.flow.planner.BaseFlowStep.prepareResources(BaseFlowStep.java:549)
at cascading.flow.planner.BaseFlowStep.prepareResources(BaseFlowStep.java:532)
at cascading.flow.planner.FlowStepJob.prepareResources(FlowStepJob.java:217)
at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:180)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:146)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:48)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2015-11-23 16:58:58,490 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping all jobs
2015-11-23 16:58:58,492 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping: (1/1) ...dFails/preparereadtapfail
2015-11-23 16:58:58,492 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopped all jobs
2015-11-23 16:58:58,493 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.005
2015-11-23 16:58:58,494 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testPrepareResourceForReadFails/preparereadtapfail
2015-11-23 16:58:58,506 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:58:58,506 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:58:58,528 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.018
2015-11-23 16:58:58,529 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:58:58,529 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:58:58,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:58:58,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:58:58,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: ["TextDelimited[['ip', 'count']]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testCommitResourceFails/committapfail"]
2015-11-23 16:58:58,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:58:58,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:58:58,548 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:58:58,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...sourceFails/committapfail
2015-11-23 16:58:58,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted hadoop job: job_1448269016123_0008
2015-11-23 16:58:58,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0008/
2015-11-23 16:59:12,627 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0008_01_000004 is : 143
2015-11-23 16:59:12,657 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0008_01_000005 is : 143
2015-11-23 16:59:12,683 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0008_01_000002 is : 143
2015-11-23 16:59:12,719 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0008_01_000003 is : 143
2015-11-23 16:59:17,506 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0008_01_000006 is : 143
2015-11-23 16:59:18,650 ERROR flow.Flow (BaseFlow.java:logError(1469)) - [test] unable to commit sink: hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testCommitResourceFails/committapfail
java.io.IOException: failed intentionally
at cascading.tap.hadoop.HadoopTapPlatformTest$2.commitResource(HadoopTapPlatformTest.java:400)
at cascading.tap.hadoop.HadoopTapPlatformTest$2.commitResource(HadoopTapPlatformTest.java:396)
at cascading.flow.planner.BaseFlowStep.commitResource(BaseFlowStep.java:613)
at cascading.flow.planner.BaseFlowStep.commitSinks(BaseFlowStep.java:601)
at cascading.flow.planner.FlowStepJob.blockOnJob(FlowStepJob.java:297)
at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:184)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:146)
at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:48)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2015-11-23 16:59:18,677 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping all jobs
2015-11-23 16:59:18,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopping: (1/1) ...sourceFails/committapfail
2015-11-23 16:59:18,685 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] stopped all jobs
2015-11-23 16:59:18,686 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testCommitResourceFails/committapfail/_temporary
2015-11-23 16:59:18,690 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:20.138, using cpu time: 00:03.780
2015-11-23 16:59:18,690 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testCommitResourceFails/committapfail
2015-11-23 16:59:18,712 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:59:18,712 INFO planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:59:18,728 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.011
2015-11-23 16:59:18,728 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:59:18,728 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:59:18,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2015-11-23 16:59:18,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:59:18,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: ["TextDelimited[['ip', 'count']]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testCommitResource/committap"]
2015-11-23 16:59:18,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: true
2015-11-23 16:59:18,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2015-11-23 16:59:18,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2015-11-23 16:59:18,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...tCommitResource/committap
2015-11-23 16:59:19,070 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted hadoop job: job_1448269016123_0009
2015-11-23 16:59:19,070 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tracking url: http://shihc046-public.cn.ibm.com:52466/proxy/application_1448269016123_0009/
2015-11-23 16:59:33,794 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0009_01_000002 is : 143
2015-11-23 16:59:33,814 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0009_01_000003 is : 143
2015-11-23 16:59:33,832 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0009_01_000005 is : 143
2015-11-23 16:59:33,870 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0009_01_000004 is : 143
2015-11-23 16:59:38,821 WARN nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448269016123_0009_01_000006 is : 143
2015-11-23 16:59:39,995 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testCommitResource/committap/_temporary
2015-11-23 16:59:40,006 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:21.256, using cpu time: 00:03.770
2015-11-23 16:59:40,025 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testCommitResource/committap
2015-11-23 16:59:40,057 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:setupTask(105)) - setting up task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,057 INFO io.TapOutputCollector (TapOutputCollector.java:initialize(117)) - creating path: part-00000
2015-11-23 16:59:40,059 INFO io.TapOutputCollector (TapOutputCollector.java:close(150)) - closing tap collector for: /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A/part-00000
2015-11-23 16:59:40,070 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(162)) - committing task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,074 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(179)) - saved output of task 'attempt_002147483647_0000_m_000000_0' to hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A
2015-11-23 16:59:40,075 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A/_temporary
2015-11-23 16:59:40,080 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:setupTask(105)) - setting up task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,080 INFO io.TapOutputCollector (TapOutputCollector.java:initialize(117)) - creating path: part-00000
2015-11-23 16:59:40,082 INFO io.TapOutputCollector (TapOutputCollector.java:close(150)) - closing tap collector for: /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B/part-00000
2015-11-23 16:59:40,095 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(162)) - committing task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,100 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(179)) - saved output of task 'attempt_002147483647_0000_m_000000_0' to hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B
2015-11-23 16:59:40,101 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B/_temporary
2015-11-23 16:59:40,135 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:setupTask(105)) - setting up task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,135 INFO io.TapOutputCollector (TapOutputCollector.java:initialize(117)) - creating path: part-00000
2015-11-23 16:59:40,137 INFO io.TapOutputCollector (TapOutputCollector.java:close(150)) - closing tap collector for: /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A/part-00000
2015-11-23 16:59:40,149 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(162)) - committing task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,154 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(179)) - saved output of task 'attempt_002147483647_0000_m_000000_0' to hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A
2015-11-23 16:59:40,155 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A/_temporary
2015-11-23 16:59:40,159 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:setupTask(105)) - setting up task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,159 INFO io.TapOutputCollector (TapOutputCollector.java:initialize(117)) - creating path: part-00000
2015-11-23 16:59:40,161 INFO io.TapOutputCollector (TapOutputCollector.java:close(150)) - closing tap collector for: /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B/part-00000
2015-11-23 16:59:40,175 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(162)) - committing task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,180 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(179)) - saved output of task 'attempt_002147483647_0000_m_000000_0' to hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B
2015-11-23 16:59:40,180 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B/_temporary
2015-11-23 16:59:40,184 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:setupTask(105)) - setting up task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,185 INFO io.TapOutputCollector (TapOutputCollector.java:initialize(117)) - creating path: part-00000
2015-11-23 16:59:40,186 INFO io.TapOutputCollector (TapOutputCollector.java:close(150)) - closing tap collector for: /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden/part-00000
2015-11-23 16:59:40,198 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(162)) - committing task: 'attempt_002147483647_0000_m_000000_0' - hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden/_temporary/_attempt_002147483647_0000_m_000000_0
2015-11-23 16:59:40,204 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:commitTask(179)) - saved output of task 'attempt_002147483647_0000_m_000000_0' to hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden
2015-11-23 16:59:40,204 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path hdfs://localhost:48910/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden/_temporary
2015-11-23 16:59:40,257 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles
2015-11-23 16:59:40,269 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/A
2015-11-23 16:59:40,273 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles/B
2015-11-23 16:59:40,302 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/multifiles
2015-11-23 16:59:40,305 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/B
2015-11-23 16:59:40,309 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/A
2015-11-23 16:59:40,313 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/hadooptapplatform/testChildIdentifiers/hiddenfiles/.hidden