使用cosbench对ceph对象存储(s3)进行性能测试
安装cosbench
wget https://github.com/intel-cloud/cosbench/releases/download/v0.4.2.c4/0.4.2.c4.zip
# 这里下载的并非最新版，最新版有bug
yum install java-1.7.0-openjdk nmap-ncat
unzip 0.4.2.c4.zip
cd 0.4.2.c4
./start-all.sh运行成功输出
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------
======================================================
Launching osgi framwork ...
Successfully launched osgi framework!
Booting cosbench controller ...
.
Starting cosbench-log_0.4.2 [OK]
Starting cosbench-tomcat_0.4.2 [OK]
Starting cosbench-config_0.4.2 [OK]
Starting cosbench-core_0.4.2 [OK]
Starting cosbench-core-web_0.4.2 [OK]
Starting cosbench-controller_0.4.2 [OK]
Starting cosbench-controller-web_0.4.2 [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ...
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------
访问web页面http://{ip}:19088/controller/index.html
配置 workloads
COSBench 可以web 编辑workloads 也可以通过上传xml格式的配置文件识别workloads。
通过web页面配置workloads目前没看到支持s3协议， 所以得在本地手动文本编辑workloads，然后上传至web。
下载s3 workloads 的模板
wget https://raw.githubusercontent.com/intel-cloud/cosbench/master/release/conf/s3-config-sample.xml用编辑器打开并按自己配置修改里面的内容
code s3-config-sample.xml<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
<storage type="s3" config="accesskey=<accesskey>;secretkey=<scretkey>;proxyhost=<proxyhost>;proxyport=<proxyport>;endpoint=<endpoint>" />
<workflow>
<!-- 创建桶 -->
<workstage name="init">
<work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
<!-- 预写少部分数据，用于后面的读 -->
<workstage name="prepare">
<work type="prepare" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(64)KB" />
</workstage>
<!-- 正式测试部分 -->
<workstage name="main">
<work name="main" workers="8" runtime="30">
<operation type="read" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />
<operation type="write" ratio="20" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(64)KB" />
</work>
</workstage>
<!-- 清除 -->
<workstage name="cleanup">
<work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
</workstage>
<!-- 删除桶 -->
<workstage name="dispose">
<work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
</workflow>
</workload>下面是一个只用来测试读和写的workload例子
<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">
<storage type="s3" config="accesskey=<accesskey>;secretkey=<scretkey>;proxyhost=<proxyhost>;proxyport=<proxyport>;endpoint=<endpoint>" />
<workflow>
<!-- 创建桶 -->
<workstage name="init">
<work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
<!-- 预先写数据 -->
<workstage name="prepare">
<work type="prepare" workers="12" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10000);sizes=c(64)KB" />
</workstage>
<!-- 开始测试 -->
<!-- 读 -->
<workstage name="read">
<work name="read" workers="12" runtime="300">
<operation type="read" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10000)" />
</work>
</workstage>
<!-- 写 -->
<workstage name="write">
<work name="write" workers="12" runtime="300">
<operation type="write" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(10001,20000);sizes=c(64)KB" />
</work>
</workstage>
<!-- 删除数据 -->
<workstage name="cleanup">
<work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20000)" />
</workstage>
<!-- 删除桶 -->
<workstage name="dispose">
<work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
</workstage>
</workflow>
</workload>点击submit new workloads按钮上传已经修改好的xml文件
测试结果
- Avg-ResTime 响应平均时间
- Avg-ProcTime 平均处理时间
- Throughput：吞吐量，也就是我们常说的TPS
- bandwith：带宽
- succ-ratio :成功数
其他性能测试工具
华为开源的obscmdbench https://github.com/huaweicloud-obs/obscmdbench