Class cascading.MergePipesPlatformTest

all > cascading > MergePipesPlatformTest
21

tests
0

failures
0

ignored
9m50.22s

duration
100%

successful

Tests

Test Duration Result
testHashJoinMergeIntoHashJoinAccumulated 35.045s passed
testHashJoinMergeIntoHashJoinAccumulatedAccumulated 50.752s passed
testHashJoinMergeIntoHashJoinAccumulatedAccumulatedMerge 52.664s passed
testHashJoinMergeIntoHashJoinAccumulatedStreamed 51.795s passed
testHashJoinMergeIntoHashJoinAccumulatedStreamedMerge 50.509s passed
testHashJoinMergeIntoHashJoinStreamed 16.274s passed
testHashJoinMergeIntoHashJoinStreamedAccumulated 33.279s passed
testHashJoinMergeIntoHashJoinStreamedAccumulatedMerge 32.819s passed
testHashJoinMergeIntoHashJoinStreamedStreamed 17.702s passed
testHashJoinMergeIntoHashJoinStreamedStreamedMerge 18.440s passed
testMergeIntoHashJoinAccumulated 17.546s passed
testMergeIntoHashJoinStreamed 18.054s passed
testSameSourceMergeThreeChainGroup 41.359s passed
testSimpleMerge 37.774s passed
testSimpleMergeFail 0.025s passed
testSimpleMergeThree 19.191s passed
testSimpleMergeThreeChain 19.742s passed
testSimpleMergeThreeChainCoGroup 23.246s passed
testSimpleMergeThreeChainGroup 22.097s passed
testSplitSameSourceMerged 16.358s passed
testSplitSameSourceMergedComplex 15.551s passed

Standard output

2015-11-23 16:21:06,246 INFO  platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2015-11-23 16:21:06,261 INFO  platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.MergePipesPlatformTest, with platform: hadoop2-mr1
2015-11-23 16:21:06,275 INFO  hadoop2.Hadoop2MR1Platform (Hadoop2MR1Platform.java:setUp(136)) - using cluster
2015-11-23 16:21:06,895 WARN  util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Formatting using clusterid: testClusterID
2015-11-23 16:21:07,494 WARN  impl.MetricsConfig (MetricsConfig.java:loadFirst(125)) - Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
2015-11-23 16:21:11,495 WARN  containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:21:11,701 WARN  containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:21:11,892 WARN  containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:21:12,084 WARN  containermanager.AuxServices (AuxServices.java:serviceInit(130)) - The Auxilurary Service named 'mapreduce_shuffle' in the configuration is for class org.apache.hadoop.mapred.ShuffleHandler which has a name of 'httpshuffle'. Because these are not the same tools trying to send ServiceData and read Service Meta Data may have issues unless the refer to the name in the config.
2015-11-23 16:21:29,191 INFO  platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides: 
2015-11-23 16:21:29,379 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:21:29,380 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:21:29,422 INFO  property.AppProps (AppProps.java:getAppID(177)) - using app.id: 88C66A57CEEA456C9F72E483B42B5D6B
2015-11-23 16:21:29,515 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.061
2015-11-23 16:21:29,517 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:21:29,519 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:21:29,775 INFO  util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.2
2015-11-23 16:21:29,776 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] starting
2015-11-23 16:21:29,777 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:21:29,777 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:21:29,777 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  sink: Hfs["TextLine[['line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMerge/simplemerge"]
2015-11-23 16:21:29,778 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  parallel execution of steps is enabled: true
2015-11-23 16:21:29,778 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  executing total steps: 1
2015-11-23 16:21:29,778 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  allocating management threads: 1
2015-11-23 16:21:29,779 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] starting step: (1/1) ...stSimpleMerge/simplemerge
2015-11-23 16:21:31,671 WARN  datanode.DataNode (BlockReceiver.java:receivePacket(651)) - Slow BlockReceiver write data to disk cost:719ms (threshold=300ms)
2015-11-23 16:21:32,350 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] submitted hadoop job: job_1448266883689_0001
2015-11-23 16:21:32,350 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0001/
2015-11-23 16:21:41,965 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000003 is : 143
2015-11-23 16:21:42,010 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000002 is : 143
2015-11-23 16:21:42,572 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000006 is : 143
2015-11-23 16:21:42,615 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000007 is : 143
2015-11-23 16:21:42,657 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000004 is : 143
2015-11-23 16:21:42,680 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0001_01_000005 is : 143
2015-11-23 16:21:43,934 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMerge/simplemerge/_temporary
2015-11-23 16:21:43,939 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  completed in: 00:14.155, using cpu time: 00:04.660
2015-11-23 16:21:44,025 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMerge/simplemerge
2015-11-23 16:21:44,094 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:21:44,094 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:21:44,172 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.072
2015-11-23 16:21:44,173 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 3, nodes: 3
2015-11-23 16:21:44,174 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:21:44,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting
2015-11-23 16:21:44,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:21:44,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:21:44,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:21:44,219 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatedinterMerge"]
2015-11-23 16:21:44,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  parallel execution of steps is enabled: true
2015-11-23 16:21:44,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  executing total steps: 3
2015-11-23 16:21:44,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  allocating management threads: 3
2015-11-23 16:21:44,223 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting step: (1/3)
2015-11-23 16:21:44,317 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:21:44,318 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:21:44,318 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:21:44,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] submitted hadoop job: job_1448266883689_0002
2015-11-23 16:21:44,398 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0002/
2015-11-23 16:21:59,606 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000007 is : 143
2015-11-23 16:21:59,649 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000006 is : 143
2015-11-23 16:21:59,946 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000004 is : 143
2015-11-23 16:21:59,986 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000003 is : 143
2015-11-23 16:22:00,010 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000005 is : 143
2015-11-23 16:22:00,052 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0002_01_000002 is : 143
2015-11-23 16:22:01,222 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting step: (2/3)
2015-11-23 16:22:01,556 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] submitted hadoop job: job_1448266883689_0003
2015-11-23 16:22:01,556 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0003/
2015-11-23 16:22:16,924 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000002 is : 143
2015-11-23 16:22:16,936 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000004 is : 143
2015-11-23 16:22:16,960 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000005 is : 143
2015-11-23 16:22:17,086 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000007 is : 143
2015-11-23 16:22:17,727 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000003 is : 143
2015-11-23 16:22:18,121 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0003_01_000006 is : 143
2015-11-23 16:22:19,318 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting step: (3/3) ...condAccumulatedinterMerge
2015-11-23 16:22:19,666 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] submitted hadoop job: job_1448266883689_0004
2015-11-23 16:22:19,667 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0004/
2015-11-23 16:22:34,332 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0004_01_000004 is : 143
2015-11-23 16:22:34,390 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0004_01_000003 is : 143
2015-11-23 16:22:34,445 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0004_01_000006 is : 143
2015-11-23 16:22:34,540 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0004_01_000005 is : 143
2015-11-23 16:22:35,388 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0004_01_000002 is : 143
2015-11-23 16:22:36,660 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatedinterMerge/_temporary
2015-11-23 16:22:36,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  completed in: 00:52.434, using cpu time: 00:15.370
2015-11-23 16:22:36,699 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatedinterMerge
2015-11-23 16:22:36,734 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:22:36,734 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:22:36,768 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.029
2015-11-23 16:22:36,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:22:36,769 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:22:36,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting
2015-11-23 16:22:36,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:22:36,786 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:22:36,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:22:36,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamedinterMerge"]
2015-11-23 16:22:36,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  parallel execution of steps is enabled: true
2015-11-23 16:22:36,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  executing total steps: 1
2015-11-23 16:22:36,787 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  allocating management threads: 1
2015-11-23 16:22:36,789 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting step: (1/1) ...dsecondStreamedinterMerge
2015-11-23 16:22:37,120 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] submitted hadoop job: job_1448266883689_0005
2015-11-23 16:22:37,120 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0005/
2015-11-23 16:22:52,890 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0005_01_000004 is : 143
2015-11-23 16:22:52,919 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0005_01_000006 is : 143
2015-11-23 16:22:52,938 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0005_01_000002 is : 143
2015-11-23 16:22:53,002 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0005_01_000005 is : 143
2015-11-23 16:22:53,924 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0005_01_000007 is : 143
2015-11-23 16:22:55,102 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamedinterMerge/_temporary
2015-11-23 16:22:55,108 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  completed in: 00:18.314, using cpu time: 00:05.150
2015-11-23 16:22:55,140 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamedinterMerge
2015-11-23 16:22:55,172 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:22:55,172 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:22:55,199 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.022
2015-11-23 16:22:55,199 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:22:55,199 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:22:55,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] starting
2015-11-23 16:22:55,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:22:55,217 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:22:55,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:22:55,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThree/simplemergethree"]
2015-11-23 16:22:55,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  parallel execution of steps is enabled: true
2015-11-23 16:22:55,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  executing total steps: 1
2015-11-23 16:22:55,218 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  allocating management threads: 1
2015-11-23 16:22:55,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] starting step: (1/1) ...rgeThree/simplemergethree
2015-11-23 16:22:55,559 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] submitted hadoop job: job_1448266883689_0006
2015-11-23 16:22:55,560 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0006/
2015-11-23 16:23:11,978 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000003 is : 143
2015-11-23 16:23:12,017 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000008 is : 143
2015-11-23 16:23:12,254 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000006 is : 143
2015-11-23 16:23:12,261 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000010 is : 143
2015-11-23 16:23:12,340 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000004 is : 143
2015-11-23 16:23:12,371 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000005 is : 143
2015-11-23 16:23:12,485 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000007 is : 143
2015-11-23 16:23:13,063 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0006_01_000009 is : 143
2015-11-23 16:23:14,287 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThree/simplemergethree/_temporary
2015-11-23 16:23:14,291 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge]  completed in: 00:19.068, using cpu time: 00:07.130
2015-11-23 16:23:14,326 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMergeThree/simplemergethree
2015-11-23 16:23:14,364 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:23:14,364 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:23:14,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.028
2015-11-23 16:23:14,397 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:23:14,398 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:23:14,415 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] starting
2015-11-23 16:23:14,415 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamed/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamednoInterMerge"]
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  parallel execution of steps is enabled: true
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  executing total steps: 1
2015-11-23 16:23:14,416 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  allocating management threads: 1
2015-11-23 16:23:14,417 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] starting step: (1/1) ...econdStreamednoInterMerge
2015-11-23 16:23:14,555 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] submitted hadoop job: job_1448266883689_0007
2015-11-23 16:23:14,555 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0007/
2015-11-23 16:23:29,689 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0007_01_000004 is : 143
2015-11-23 16:23:29,706 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0007_01_000002 is : 143
2015-11-23 16:23:29,774 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0007_01_000006 is : 143
2015-11-23 16:23:30,758 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0007_01_000003 is : 143
2015-11-23 16:23:30,779 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0007_01_000005 is : 143
2015-11-23 16:23:32,003 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamed/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamednoInterMerge/_temporary
2015-11-23 16:23:32,008 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge1*offset*offset]  completed in: 00:17.587, using cpu time: 00:05.120
2015-11-23 16:23:32,034 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedStreamed/multihashjoinintomergeintohashjoinfirstStreamedsecondStreamednoInterMerge
2015-11-23 16:23:32,068 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:23:32,068 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:23:32,090 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.018
2015-11-23 16:23:32,091 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:23:32,091 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChain/simplemergethreechain"]
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  parallel execution of steps is enabled: true
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  executing total steps: 1
2015-11-23 16:23:32,106 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  allocating management threads: 1
2015-11-23 16:23:32,108 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting step: (1/1) ...ain/simplemergethreechain
2015-11-23 16:23:32,249 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] submitted hadoop job: job_1448266883689_0008
2015-11-23 16:23:32,250 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0008/
2015-11-23 16:23:49,477 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000004 is : 143
2015-11-23 16:23:49,497 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000005 is : 143
2015-11-23 16:23:49,793 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000003 is : 143
2015-11-23 16:23:49,847 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000010 is : 143
2015-11-23 16:23:50,409 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000008 is : 143
2015-11-23 16:23:50,529 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000002 is : 143
2015-11-23 16:23:50,546 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000006 is : 143
2015-11-23 16:23:50,564 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000009 is : 143
2015-11-23 16:23:50,583 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0008_01_000007 is : 143
2015-11-23 16:23:51,733 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChain/simplemergethreechain/_temporary
2015-11-23 16:23:51,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  completed in: 00:19.625, using cpu time: 00:07.190
2015-11-23 16:23:51,772 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChain/simplemergethreechain
2015-11-23 16:23:51,810 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:23:51,811 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:23:51,843 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.028
2015-11-23 16:23:51,844 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 2
2015-11-23 16:23:51,844 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:23:51,866 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulated/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatednoInterMerge"]
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  parallel execution of steps is enabled: true
2015-11-23 16:23:51,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  executing total steps: 2
2015-11-23 16:23:51,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  allocating management threads: 2
2015-11-23 16:23:51,870 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting step: (1/2)
2015-11-23 16:23:52,208 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] submitted hadoop job: job_1448266883689_0009
2015-11-23 16:23:52,208 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0009/
2015-11-23 16:24:06,646 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000007 is : 143
2015-11-23 16:24:06,689 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000003 is : 143
2015-11-23 16:24:06,744 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000004 is : 143
2015-11-23 16:24:06,792 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000006 is : 143
2015-11-23 16:24:07,162 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000002 is : 143
2015-11-23 16:24:07,704 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0009_01_000005 is : 143
2015-11-23 16:24:08,861 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting step: (2/2) ...ndAccumulatednoInterMerge
2015-11-23 16:24:09,209 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] submitted hadoop job: job_1448266883689_0010
2015-11-23 16:24:09,209 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0010/
2015-11-23 16:24:23,564 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0010_01_000002 is : 143
2015-11-23 16:24:23,648 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0010_01_000003 is : 143
2015-11-23 16:24:23,832 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0010_01_000004 is : 143
2015-11-23 16:24:23,846 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0010_01_000006 is : 143
2015-11-23 16:24:23,863 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0010_01_000005 is : 143
2015-11-23 16:24:25,027 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulated/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatednoInterMerge/_temporary
2015-11-23 16:24:25,031 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  completed in: 00:33.159, using cpu time: 00:10.290
2015-11-23 16:24:25,056 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulated/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatednoInterMerge
2015-11-23 16:24:25,101 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:24:25,102 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:24:25,122 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.017
2015-11-23 16:24:25,123 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:24:25,123 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:24:25,133 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] starting
2015-11-23 16:24:25,133 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:24:25,133 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSplitSameSourceMerged/splitsourcemerged"]
2015-11-23 16:24:25,133 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  parallel execution of steps is enabled: true
2015-11-23 16:24:25,133 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  executing total steps: 1
2015-11-23 16:24:25,134 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  allocating management threads: 1
2015-11-23 16:24:25,135 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] starting step: (1/1) ...eMerged/splitsourcemerged
2015-11-23 16:24:25,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] submitted hadoop job: job_1448266883689_0011
2015-11-23 16:24:25,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0011/
2015-11-23 16:24:39,154 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0011_01_000004 is : 143
2015-11-23 16:24:39,174 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0011_01_000003 is : 143
2015-11-23 16:24:39,191 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0011_01_000005 is : 143
2015-11-23 16:24:40,204 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0011_01_000002 is : 143
2015-11-23 16:24:41,392 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSplitSameSourceMerged/splitsourcemerged/_temporary
2015-11-23 16:24:41,396 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged]  completed in: 00:16.257, using cpu time: 00:03.210
2015-11-23 16:24:41,417 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSplitSameSourceMerged/splitsourcemerged
2015-11-23 16:24:41,448 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:24:41,448 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:24:41,474 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.022
2015-11-23 16:24:41,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:24:41,475 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:24:41,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting
2015-11-23 16:24:41,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:24:41,509 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:24:41,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:24:41,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamed/hashjoinintomergeintohashjoinstreamed"]
2015-11-23 16:24:41,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  parallel execution of steps is enabled: true
2015-11-23 16:24:41,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  executing total steps: 1
2015-11-23 16:24:41,510 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  allocating management threads: 1
2015-11-23 16:24:41,511 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting step: (1/1) ...mergeintohashjoinstreamed
2015-11-23 16:24:41,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] submitted hadoop job: job_1448266883689_0012
2015-11-23 16:24:41,638 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0012/
2015-11-23 16:24:55,610 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000003 is : 143
2015-11-23 16:24:55,730 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000002 is : 143
2015-11-23 16:24:55,950 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000007 is : 143
2015-11-23 16:24:55,980 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000006 is : 143
2015-11-23 16:24:56,057 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000005 is : 143
2015-11-23 16:24:56,451 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0012_01_000004 is : 143
2015-11-23 16:24:57,664 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamed/hashjoinintomergeintohashjoinstreamed/_temporary
2015-11-23 16:24:57,667 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  completed in: 00:16.154, using cpu time: 00:05.390
2015-11-23 16:24:57,692 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamed/hashjoinintomergeintohashjoinstreamed
2015-11-23 16:24:57,717 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:24:57,717 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:24:57,725 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] executed rule registry: MapReduceHadoopRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2015-11-23 16:24:57,725 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge] rule registry: MapReduceHadoopRuleRegistry, found assembly to be malformed
2015-11-23 16:24:57,725 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMergeFail/simplemergefail
2015-11-23 16:24:57,739 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:24:57,739 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:24:57,759 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.016
2015-11-23 16:24:57,760 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:24:57,760 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainGroup/simplemergethreechaingroup"]
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  parallel execution of steps is enabled: true
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  executing total steps: 1
2015-11-23 16:24:57,775 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  allocating management threads: 1
2015-11-23 16:24:57,777 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting step: (1/1) ...implemergethreechaingroup
2015-11-23 16:24:57,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] submitted hadoop job: job_1448266883689_0013
2015-11-23 16:24:57,874 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0013/
2015-11-23 16:25:14,200 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000010 is : 143
2015-11-23 16:25:15,098 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000005 is : 143
2015-11-23 16:25:15,147 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000002 is : 143
2015-11-23 16:25:15,278 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000008 is : 143
2015-11-23 16:25:15,367 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000004 is : 143
2015-11-23 16:25:15,408 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000007 is : 143
2015-11-23 16:25:15,498 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000006 is : 143
2015-11-23 16:25:15,531 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000009 is : 143
2015-11-23 16:25:18,601 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0013_01_000011 is : 143
2015-11-23 16:25:19,794 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainGroup/simplemergethreechaingroup/_temporary
2015-11-23 16:25:19,798 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  completed in: 00:22.018, using cpu time: 00:07.470
2015-11-23 16:25:19,820 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainGroup/simplemergethreechaingroup
2015-11-23 16:25:19,837 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:25:19,837 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:25:19,867 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.027
2015-11-23 16:25:19,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 3, nodes: 3
2015-11-23 16:25:19,868 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:25:19,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting
2015-11-23 16:25:19,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:25:19,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:25:19,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:25:19,890 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamed/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamednoInterMerge"]
2015-11-23 16:25:19,891 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  parallel execution of steps is enabled: true
2015-11-23 16:25:19,891 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  executing total steps: 3
2015-11-23 16:25:19,891 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  allocating management threads: 3
2015-11-23 16:25:19,893 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting step: (1/3)
2015-11-23 16:25:20,223 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] submitted hadoop job: job_1448266883689_0014
2015-11-23 16:25:20,223 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0014/
2015-11-23 16:25:34,770 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000004 is : 143
2015-11-23 16:25:34,891 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000007 is : 143
2015-11-23 16:25:34,949 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000006 is : 143
2015-11-23 16:25:34,964 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000003 is : 143
2015-11-23 16:25:34,983 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000005 is : 143
2015-11-23 16:25:35,908 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0014_01_000002 is : 143
2015-11-23 16:25:37,067 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting step: (2/3)
2015-11-23 16:25:37,174 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] submitted hadoop job: job_1448266883689_0015
2015-11-23 16:25:37,174 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0015/
2015-11-23 16:25:52,279 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0015_01_000006 is : 143
2015-11-23 16:25:52,293 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0015_01_000003 is : 143
2015-11-23 16:25:52,316 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0015_01_000005 is : 143
2015-11-23 16:25:52,332 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0015_01_000004 is : 143
2015-11-23 16:25:53,280 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0015_01_000002 is : 143
2015-11-23 16:25:54,420 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] starting step: (3/3) ...econdStreamednoInterMerge
2015-11-23 16:25:54,553 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] submitted hadoop job: job_1448266883689_0016
2015-11-23 16:25:54,553 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0016/
2015-11-23 16:26:09,333 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0016_01_000002 is : 143
2015-11-23 16:26:09,352 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0016_01_000004 is : 143
2015-11-23 16:26:09,372 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0016_01_000005 is : 143
2015-11-23 16:26:09,466 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0016_01_000003 is : 143
2015-11-23 16:26:10,388 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0016_01_000006 is : 143
2015-11-23 16:26:11,573 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamed/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamednoInterMerge/_temporary
2015-11-23 16:26:11,582 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge1*offset]  completed in: 00:51.680, using cpu time: 00:14.770
2015-11-23 16:26:11,607 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamed/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamednoInterMerge
2015-11-23 16:26:11,638 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:26:11,639 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:26:11,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.015
2015-11-23 16:26:11,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:26:11,657 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:26:11,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting
2015-11-23 16:26:11,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:26:11,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:26:11,670 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:26:11,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinStreamed/mergeintohashjoinstreamed"]
2015-11-23 16:26:11,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  parallel execution of steps is enabled: true
2015-11-23 16:26:11,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  executing total steps: 1
2015-11-23 16:26:11,671 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  allocating management threads: 1
2015-11-23 16:26:11,673 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting step: (1/1) ...mergeintohashjoinstreamed
2015-11-23 16:26:11,811 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] submitted hadoop job: job_1448266883689_0017
2015-11-23 16:26:11,812 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0017/
2015-11-23 16:26:27,388 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000003 is : 143
2015-11-23 16:26:27,843 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000004 is : 143
2015-11-23 16:26:27,897 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000006 is : 143
2015-11-23 16:26:27,912 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000007 is : 143
2015-11-23 16:26:27,967 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000005 is : 143
2015-11-23 16:26:28,445 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0017_01_000002 is : 143
2015-11-23 16:26:29,639 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinStreamed/mergeintohashjoinstreamed/_temporary
2015-11-23 16:26:29,642 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  completed in: 00:17.968, using cpu time: 00:05.090
2015-11-23 16:26:29,661 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinStreamed/mergeintohashjoinstreamed
2015-11-23 16:26:29,691 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:26:29,692 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:26:29,714 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.018
2015-11-23 16:26:29,714 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 2
2015-11-23 16:26:29,714 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:26:29,733 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainCoGroup/simplemergethreechaincogroup"]
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  parallel execution of steps is enabled: true
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  executing total steps: 1
2015-11-23 16:26:29,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  allocating management threads: 1
2015-11-23 16:26:29,736 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] starting step: (1/1) ...plemergethreechaincogroup
2015-11-23 16:26:30,067 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] submitted hadoop job: job_1448266883689_0018
2015-11-23 16:26:30,068 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0018/
2015-11-23 16:26:45,897 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000010 is : 143
2015-11-23 16:26:46,517 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000003 is : 143
2015-11-23 16:26:46,737 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000007 is : 143
2015-11-23 16:26:46,746 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000005 is : 143
2015-11-23 16:26:46,764 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000004 is : 143
2015-11-23 16:26:46,802 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000002 is : 143
2015-11-23 16:26:46,853 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000009 is : 143
2015-11-23 16:26:46,873 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000008 is : 143
2015-11-23 16:26:51,738 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0018_01_000011 is : 143
2015-11-23 16:26:52,892 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainCoGroup/simplemergethreechaincogroup/_temporary
2015-11-23 16:26:52,896 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge*offset]  completed in: 00:23.157, using cpu time: 00:07.760
2015-11-23 16:26:52,915 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSimpleMergeThreeChainCoGroup/simplemergethreechaincogroup
2015-11-23 16:26:52,939 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:26:52,939 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:26:52,962 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.020
2015-11-23 16:26:52,963 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 4
2015-11-23 16:26:52,963 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:26:52,976 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting
2015-11-23 16:26:52,976 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:26:52,976 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSameSourceMergeThreeChainGroup/samemergethreechaingroup"]
2015-11-23 16:26:52,976 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  parallel execution of steps is enabled: true
2015-11-23 16:26:52,976 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  executing total steps: 2
2015-11-23 16:26:52,977 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  allocating management threads: 2
2015-11-23 16:26:52,979 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting step: (1/2)
2015-11-23 16:26:53,101 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] submitted hadoop job: job_1448266883689_0019
2015-11-23 16:26:53,101 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0019/
2015-11-23 16:27:07,043 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000006 is : 143
2015-11-23 16:27:07,545 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000003 is : 143
2015-11-23 16:27:07,573 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000002 is : 143
2015-11-23 16:27:07,591 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000005 is : 143
2015-11-23 16:27:07,620 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000004 is : 143
2015-11-23 16:27:12,152 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0019_01_000007 is : 143
2015-11-23 16:27:13,259 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] starting step: (2/2) .../samemergethreechaingroup
2015-11-23 16:27:13,585 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] submitted hadoop job: job_1448266883689_0020
2015-11-23 16:27:13,585 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0020/
2015-11-23 16:27:28,583 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000002 is : 143
2015-11-23 16:27:28,633 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000003 is : 143
2015-11-23 16:27:28,898 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000006 is : 143
2015-11-23 16:27:28,946 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000005 is : 143
2015-11-23 16:27:28,991 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000004 is : 143
2015-11-23 16:27:29,055 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000007 is : 143
2015-11-23 16:27:33,045 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0020_01_000008 is : 143
2015-11-23 16:27:34,243 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSameSourceMergeThreeChainGroup/samemergethreechaingroup/_temporary
2015-11-23 16:27:34,258 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge+offset]  completed in: 00:41.264, using cpu time: 00:09.700
2015-11-23 16:27:34,274 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSameSourceMergeThreeChainGroup/samemergethreechaingroup
2015-11-23 16:27:34,300 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:27:34,300 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:27:34,337 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.034
2015-11-23 16:27:34,338 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 3, nodes: 3
2015-11-23 16:27:34,338 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:27:34,355 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamedinterMerge"]
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  parallel execution of steps is enabled: true
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  executing total steps: 3
2015-11-23 16:27:34,356 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  allocating management threads: 3
2015-11-23 16:27:34,358 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting step: (1/3)
2015-11-23 16:27:34,682 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] submitted hadoop job: job_1448266883689_0021
2015-11-23 16:27:34,682 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0021/
2015-11-23 16:27:48,574 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000002 is : 143
2015-11-23 16:27:48,920 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000006 is : 143
2015-11-23 16:27:49,046 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000007 is : 143
2015-11-23 16:27:49,609 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000003 is : 143
2015-11-23 16:27:49,953 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000005 is : 143
2015-11-23 16:27:49,974 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0021_01_000004 is : 143
2015-11-23 16:27:51,119 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting step: (2/3)
2015-11-23 16:27:51,260 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] submitted hadoop job: job_1448266883689_0022
2015-11-23 16:27:51,260 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0022/
2015-11-23 16:28:05,542 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0022_01_000002 is : 143
2015-11-23 16:28:05,954 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0022_01_000007 is : 143
2015-11-23 16:28:06,027 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0022_01_000006 is : 143
2015-11-23 16:28:06,049 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0022_01_000005 is : 143
2015-11-23 16:28:06,114 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0022_01_000004 is : 143
2015-11-23 16:28:07,276 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] starting step: (3/3) ...dsecondStreamedinterMerge
2015-11-23 16:28:07,593 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] submitted hadoop job: job_1448266883689_0023
2015-11-23 16:28:07,593 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0023/
2015-11-23 16:28:22,449 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000002 is : 143
2015-11-23 16:28:22,451 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000006 is : 143
2015-11-23 16:28:22,482 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000007 is : 143
2015-11-23 16:28:22,537 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000005 is : 143
2015-11-23 16:28:22,870 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000004 is : 143
2015-11-23 16:28:23,534 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0023_01_000003 is : 143
2015-11-23 16:28:24,740 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamedinterMerge/_temporary
2015-11-23 16:28:24,750 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merge2*offset]  completed in: 00:50.381, using cpu time: 00:16.070
2015-11-23 16:28:24,778 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedStreamedMerge/multihashjoinintomergeintohashjoinfirstAccumulatedsecondStreamedinterMerge
2015-11-23 16:28:24,809 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:28:24,809 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:28:24,823 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.011
2015-11-23 16:28:24,824 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:28:24,824 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:28:24,833 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] starting
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinAccumulated/mergeintohashjoinaccumulated"]
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  parallel execution of steps is enabled: true
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  executing total steps: 1
2015-11-23 16:28:24,834 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  allocating management threads: 1
2015-11-23 16:28:24,836 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] starting step: (1/1) ...geintohashjoinaccumulated
2015-11-23 16:28:24,924 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] submitted hadoop job: job_1448266883689_0024
2015-11-23 16:28:24,924 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0024/
2015-11-23 16:28:40,220 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0024_01_000005 is : 143
2015-11-23 16:28:40,292 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0024_01_000004 is : 143
2015-11-23 16:28:40,396 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0024_01_000006 is : 143
2015-11-23 16:28:41,126 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0024_01_000003 is : 143
2015-11-23 16:28:42,292 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinAccumulated/mergeintohashjoinaccumulated/_temporary
2015-11-23 16:28:42,297 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  completed in: 00:17.458, using cpu time: 00:04.210
2015-11-23 16:28:42,324 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testMergeIntoHashJoinAccumulated/mergeintohashjoinaccumulated
2015-11-23 16:28:42,352 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:28:42,353 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:28:42,371 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.015
2015-11-23 16:28:42,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 1, nodes: 1
2015-11-23 16:28:42,372 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:28:42,383 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] starting
2015-11-23 16:28:42,383 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/apache.10.txt"]
2015-11-23 16:28:42,383 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSplitSameSourceMergedComplex/splitsourcemergedcomplex"]
2015-11-23 16:28:42,384 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  parallel execution of steps is enabled: true
2015-11-23 16:28:42,384 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  executing total steps: 1
2015-11-23 16:28:42,384 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  allocating management threads: 1
2015-11-23 16:28:42,385 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] starting step: (1/1) .../splitsourcemergedcomplex
2015-11-23 16:28:42,700 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] submitted hadoop job: job_1448266883689_0025
2015-11-23 16:28:42,700 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0025/
2015-11-23 16:28:56,500 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0025_01_000004 is : 143
2015-11-23 16:28:56,592 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0025_01_000003 is : 143
2015-11-23 16:28:56,644 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0025_01_000005 is : 143
2015-11-23 16:28:56,689 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0025_01_000002 is : 143
2015-11-23 16:28:57,852 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testSplitSameSourceMergedComplex/splitsourcemergedcomplex/_temporary
2015-11-23 16:28:57,857 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [merged-second]  completed in: 00:15.468, using cpu time: 00:03.430
2015-11-23 16:28:57,878 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testSplitSameSourceMergedComplex/splitsourcemergedcomplex
2015-11-23 16:28:57,908 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:28:57,908 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:28:57,930 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.018
2015-11-23 16:28:57,931 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 2
2015-11-23 16:28:57,931 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:28:57,949 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] starting
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulated/hashjoinintomergeintohashjoinaccumulated"]
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  parallel execution of steps is enabled: true
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  executing total steps: 2
2015-11-23 16:28:57,950 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  allocating management threads: 2
2015-11-23 16:28:57,952 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] starting step: (1/2)
2015-11-23 16:28:58,005 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:28:58,006 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:28:58,006 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:28:58,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] submitted hadoop job: job_1448266883689_0026
2015-11-23 16:28:58,262 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0026/
2015-11-23 16:28:59,364 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:28:59,365 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:28:59,365 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:28:59,365 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable:  unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:28:59,365 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:28:59,365 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:28:59,365 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:28:59,366 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable:  unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:29:13,936 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000005 is : 143
2015-11-23 16:29:13,982 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000003 is : 143
2015-11-23 16:29:13,986 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000006 is : 143
2015-11-23 16:29:14,002 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000002 is : 143
2015-11-23 16:29:14,040 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000004 is : 143
2015-11-23 16:29:14,990 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0026_01_000007 is : 143
2015-11-23 16:29:16,146 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] starting step: (2/2) ...geintohashjoinaccumulated
2015-11-23 16:29:16,266 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] submitted hadoop job: job_1448266883689_0027
2015-11-23 16:29:16,266 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0027/
2015-11-23 16:29:30,678 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0027_01_000002 is : 143
2015-11-23 16:29:30,800 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0027_01_000004 is : 143
2015-11-23 16:29:30,923 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0027_01_000006 is : 143
2015-11-23 16:29:30,942 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0027_01_000005 is : 143
2015-11-23 16:29:31,725 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0027_01_000003 is : 143
2015-11-23 16:29:32,892 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulated/hashjoinintomergeintohashjoinaccumulated/_temporary
2015-11-23 16:29:32,899 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge]  completed in: 00:34.941, using cpu time: 00:09.830
2015-11-23 16:29:32,924 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulated/hashjoinintomergeintohashjoinaccumulated
2015-11-23 16:29:32,946 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:29:32,946 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:29:32,981 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.031
2015-11-23 16:29:32,981 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 3, nodes: 3
2015-11-23 16:29:32,981 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] starting
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulated/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatednoInterMerge"]
2015-11-23 16:29:33,000 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  parallel execution of steps is enabled: true
2015-11-23 16:29:33,001 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  executing total steps: 3
2015-11-23 16:29:33,001 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  allocating management threads: 3
2015-11-23 16:29:33,003 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] starting step: (1/3)
2015-11-23 16:29:33,326 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] submitted hadoop job: job_1448266883689_0028
2015-11-23 16:29:33,327 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0028/
2015-11-23 16:29:48,549 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000004 is : 143
2015-11-23 16:29:48,572 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000005 is : 143
2015-11-23 16:29:48,584 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000003 is : 143
2015-11-23 16:29:49,602 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000006 is : 143
2015-11-23 16:29:49,618 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000002 is : 143
2015-11-23 16:29:49,633 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0028_01_000007 is : 143
2015-11-23 16:29:50,767 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] starting step: (2/3)
2015-11-23 16:29:51,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] submitted hadoop job: job_1448266883689_0029
2015-11-23 16:29:51,279 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0029/
2015-11-23 16:30:05,237 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0029_01_000002 is : 143
2015-11-23 16:30:05,303 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0029_01_000003 is : 143
2015-11-23 16:30:05,576 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0029_01_000004 is : 143
2015-11-23 16:30:05,622 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0029_01_000005 is : 143
2015-11-23 16:30:05,641 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0029_01_000006 is : 143
2015-11-23 16:30:06,762 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] starting step: (3/3) ...ndAccumulatednoInterMerge
2015-11-23 16:30:06,836 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:30:06,836 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:30:06,836 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=true) All required storage types are unavailable:  unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:30:07,095 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] submitted hadoop job: job_1448266883689_0030
2015-11-23 16:30:07,095 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0030/
2015-11-23 16:30:08,374 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:30:08,375 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:30:08,375 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:30:08,375 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable:  unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:30:08,375 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:30:08,375 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) For more information, please enable DEBUG log level on org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
2015-11-23 16:30:08,375 WARN  protocol.BlockStoragePolicy (BlockStoragePolicy.java:chooseStorageTypes(160)) - Failed to place enough replicas: expected size is 1 but only 0 storage types can be selected (replication=4, selected=[], unavailable=[DISK, ARCHIVE], removed=[DISK], policy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]})
2015-11-23 16:30:08,376 WARN  blockmanagement.BlockPlacementPolicy (BlockPlacementPolicyDefault.java:chooseTarget(382)) - Failed to place enough replicas, still in need of 1 to reach 4 (unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}, newBlock=false) All required storage types are unavailable:  unavailableStorages=[DISK, ARCHIVE], storagePolicy=BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
2015-11-23 16:30:21,545 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0030_01_000002 is : 143
2015-11-23 16:30:21,551 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0030_01_000006 is : 143
2015-11-23 16:30:21,574 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0030_01_000003 is : 143
2015-11-23 16:30:21,650 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0030_01_000005 is : 143
2015-11-23 16:30:22,465 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0030_01_000004 is : 143
2015-11-23 16:30:23,643 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulated/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatednoInterMerge/_temporary
2015-11-23 16:30:23,650 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*offset*merge1]  completed in: 00:50.641, using cpu time: 00:14.810
2015-11-23 16:30:23,674 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinAccumulatedAccumulated/multihashjoinintomergeintohashjoinfirstAccumulatedsecondAccumulatednoInterMerge
2015-11-23 16:30:23,707 INFO  util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2015-11-23 16:30:23,707 INFO  planner.HadoopPlanner (HadoopPlanner.java:initialize(174)) - using application jar: /root/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-mr1/3.0.2/a7e0a109fe1f98639fc92e32694f202ce66d81a8/cascading-hadoop2-mr1-3.0.2.jar
2015-11-23 16:30:23,733 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] executed rule registry: MapReduceHadoopRuleRegistry, completed as: SUCCESS, in: 00:00.022
2015-11-23 16:30:23,733 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] rule registry: MapReduceHadoopRuleRegistry, supports assembly with steps: 2, nodes: 2
2015-11-23 16:30:23,734 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] rule registry: MapReduceHadoopRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2015-11-23 16:30:23,753 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower-offset.txt"]
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/lower.txt"]
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/classes/test/data/upper.txt"]
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatedinterMerge"]
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  parallel execution of steps is enabled: true
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  executing total steps: 2
2015-11-23 16:30:23,754 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  allocating management threads: 2
2015-11-23 16:30:23,757 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting step: (1/2)
2015-11-23 16:30:24,047 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] submitted hadoop job: job_1448266883689_0031
2015-11-23 16:30:24,047 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0031/
2015-11-23 16:30:38,229 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000003 is : 143
2015-11-23 16:30:38,243 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000002 is : 143
2015-11-23 16:30:38,661 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000005 is : 143
2015-11-23 16:30:38,695 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000006 is : 143
2015-11-23 16:30:38,741 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000007 is : 143
2015-11-23 16:30:38,761 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0031_01_000004 is : 143
2015-11-23 16:30:39,909 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] starting step: (2/2) ...condAccumulatedinterMerge
2015-11-23 16:30:40,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] submitted hadoop job: job_1448266883689_0032
2015-11-23 16:30:40,220 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2] tracking url: http://shihc046-public.cn.ibm.com:51108/proxy/application_1448266883689_0032/
2015-11-23 16:30:54,246 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0032_01_000006 is : 143
2015-11-23 16:30:54,315 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0032_01_000002 is : 143
2015-11-23 16:30:54,369 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0032_01_000003 is : 143
2015-11-23 16:30:54,541 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0032_01_000005 is : 143
2015-11-23 16:30:55,301 WARN  nodemanager.DefaultContainerExecutor (DefaultContainerExecutor.java:launchContainer(223)) - Exit code from container container_1448266883689_0032_01_000004 is : 143
2015-11-23 16:30:56,464 INFO  util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output/hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatedinterMerge/_temporary
2015-11-23 16:30:56,471 INFO  flow.Flow (BaseFlow.java:logInfo(1433)) - [offset*merge2]  completed in: 00:32.709, using cpu time: 00:10.090
2015-11-23 16:30:56,495 INFO  cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /root/yuanbo/cascading.compatibility-3.0/biginsights-4.1/build/test/output//hadoop2-mr1/mergepipesplatform/testHashJoinMergeIntoHashJoinStreamedAccumulatedMerge/multihashjoinintomergeintohashjoinfirstStreamedsecondAccumulatedinterMerge

Standard error

Nov 23, 2015 4:21:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.hs.webapp.HsWebServices as a root resource class
Nov 23, 2015 4:21:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.hs.webapp.JAXBContextResolver as a provider class
Nov 23, 2015 4:21:23 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 23, 2015 4:21:23 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 23, 2015 4:21:24 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.hs.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:24 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:24 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.hs.webapp.HsWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
Nov 23, 2015 4:21:24 PM com.google.inject.servlet.GuiceFilter setPipeline
WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver as a provider class
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices as a root resource class
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 23, 2015 4:21:25 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:25 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:25 PM com.google.inject.servlet.GuiceFilter setPipeline
WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class
Nov 23, 2015 4:21:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:26 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:26 PM com.google.inject.servlet.GuiceFilter setPipeline
WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class
Nov 23, 2015 4:21:27 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:27 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:27 PM com.google.inject.servlet.GuiceFilter setPipeline
WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices as a root resource class
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver as a provider class
Nov 23, 2015 4:21:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:28 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.server.nodemanager.webapp.NMWebServices to GuiceManagedComponentProvider with the scope "Singleton"
Nov 23, 2015 4:21:28 PM com.google.inject.servlet.GuiceFilter setPipeline
WARNING: Multiple Servlet injectors detected. This is a warning indicating that you have more than one GuiceFilter running in your web application. If this is deliberate, you may safely ignore this message. If this is NOT deliberate however, your application may not work as expected.