其实就是从网络上copy的吧，在这里做一下记录

这个是看一下有哪些测试方式：

hadoop  jar /opt/cloudera/parcels/CDH-5.3.6-1.cdh5.3.6.p0.11/jars/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.6-tests.jar

 

测试hadoop写的速度

向HDFS文件系统中写入数据,30个文件,每个文件100MB,文件存放到/benchmarks/TestDFSIO/io_data中

hadoop  jar /opt/cloudera/parcels/CDH-5.3.6-1.cdh5.3.6.p0.11/jars/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.6-tests.jar TestDFSIO -write -nrFiles 30 -fileSize 100MB

然后查看结果：cat TestDFSIO_results.log

我的集群基准测试结果如下 做了两次，有两个不同的结果，从结果上来看，为什么变化这么大：

----- TestDFSIO ----- : write

Date & time: Thu Sep 17 16:45:03 CST 2015

Number of files: 10

Total MBytes processed: 100.0

Throughput mb/sec: 27.51031636863824

Average IO rate mb/sec: 30.240123748779297

IO rate std deviation: 8.554948120135029

Test exec time sec: 30.227

----- TestDFSIO ----- : write

Date & time: Thu Sep 17 16:49:53 CST 2015

Number of files: 30

Total MBytes processed: 3000.0

Throughput mb/sec: 7.770168768065642

Average IO rate mb/sec: 8.027955055236816

IO rate std deviation: 1.629595948634101

Test exec time sec: 41.057

 

 

测试一下读的速度

hadoop  jar /opt/cloudera/parcels/CDH-5.3.6-1.cdh5.3.6.p0.11/jars/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.6-tests.jar TestDFSIO -read -nrFiles 30 -fileSize 100MB

结果如下：

----- TestDFSIO ----- : read

Date & time: Thu Sep 17 16:55:26 CST 2015

Number of files: 30

Total MBytes processed: 3000.0

Throughput mb/sec: 55.33115697449234

Average IO rate mb/sec: 215.3984375

IO rate std deviation: 181.40860904339297

Test exec time sec: 27.108

 

清除一下测试数据：

hadoop  jar /opt/cloudera/parcels/CDH-5.3.6-1.cdh5.3.6.p0.11/jars/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.6-tests.jar  TestDFSIO -clean