Andreas,<br><br>By saying "If I make a benchmark by writing with dd to the local (virtual disk, lying on storage cluster) of the test-VM, I get only about <span style="background-color: rgb(255, 204, 51);">20-25 MB/s.</span>" what is your block size. I guess the IOPS is really what you should concern about. The other thing is, try to assign Ethernet HBA as "direct connect" to your virtual machine.<br>
<br>Ben<br><br><br clear="all">Commit yourself to constant self-improvement<br>
<br><br><div class="gmail_quote">On Wed, Sep 22, 2010 at 11:52 AM, Andreas Erhard <span dir="ltr"><<a href="mailto:erhard@wor.net">erhard@wor.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi @all,<br>
I've got the following Openfiler HA-Cluster configuration here:<br>
Primary: VirtualMachine (on ESXi 4):<br>
2x2Ghz Intel Xeon<br>
1GB RAM<br>
Secondary: physical Machine:<br>
Intel Atom 330 1,6Ghz<br>
1GB RAM<br>
<br>
Both have two GBit interfaces, one for replication an the other one for<br>
direct access.<br>
<br>
Now I created an iSCSI volume and connected to it with another ESXi 4 as<br>
datastore and put a VM, for testing purpose, on this iSCSI-datastore.<br>
The ESXi server and the storage cluster are connected by gbit.<br>
<br>
If I make a benchmark by writing with dd to the local (virtual disk,<br>
lying on storage cluster) of the test-VM, I get only about 20-25 MB/s. I<br>
really expected atleast a writing speed of about 50MB/s.<br>
If I test writing to local disk of the storage servers, I get read: 232<br>
MB/s and write: 190 MB/s on the primary and on the secondary read: 80<br>
MB/s write: 60 MB/s.<br>
<br>
Where's the bottleneck?<br>
What's wrong with this hardware-configuration?<br>
Do you have any tipps, what could I try, to get an acceptable write<br>
perfomance of about 100-120MB/s?<br>
<br>
Here's my DRBD.conf:<br>
global {<br>
# minor-count 64;<br>
# dialog-refresh 5; # 5 seconds<br>
# disable-ip-verification;<br>
usage-count ask;<br>
}<br>
<br>
common {<br>
syncer {<br>
rate 40M;<br>
al-extents 557;<br>
}<br>
net {<br>
#after-sb-0pri discard-zero-changes;<br>
#after-sb-1pri discard-secondary;<br>
#after-sb-2pri disconnect;<br>
<br>
max-epoch-size 8192;<br>
max-buffers 8192;<br>
unplug-watermark 8192;<br>
#sndbuf-size 512k;<br>
}<br>
<br>
startup {<br>
degr-wfc-timeout 120;<br>
wfc-timeout 240;<br>
}<br>
}<br>
<br>
resource cluster_metadata {<br>
protocol C;<br>
handlers {<br>
pri-on-incon-degr "echo O > /proc/sysrq-trigger ; halt -f";<br>
pri-lost-after-sb "echo O > /proc/sysrq-trigger ; halt -f";<br>
local-io-error "echo O > /proc/sysrq-trigger ; halt -f";<br>
# outdate-peer "/usr/sbin/drbd-peer-outdater";<br>
}<br>
<br>
startup {<br>
# wfc-timeout 0;<br>
degr-wfc-timeout 120; # 2 minutes.<br>
}<br>
<br>
disk {<br>
on-io-error detach;<br>
}<br>
<br>
net {<br>
after-sb-0pri disconnect;<br>
after-sb-1pri disconnect;<br>
after-sb-2pri disconnect;<br>
rr-conflict disconnect;<br>
}<br>
<br>
syncer {<br>
al-extents 257;<br>
}<br>
<br>
on <a href="http://storage3-a.wor.net" target="_blank">storage3-a.wor.net</a> {<br>
device /dev/drbd0;<br>
disk /dev/sda2;<br>
address <a href="http://192.168.93.3:7788" target="_blank">192.168.93.3:7788</a>;<br>
meta-disk internal;<br>
}<br>
<br>
on <a href="http://storage3-b.wor.net" target="_blank">storage3-b.wor.net</a> {<br>
device /dev/drbd0;<br>
disk /dev/sda2;<br>
address <a href="http://192.168.93.4:7788" target="_blank">192.168.93.4:7788</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
resource vg0drbd {<br>
protocol C;<br>
startup {<br>
wfc-timeout 0; ## Infinite!<br>
degr-wfc-timeout 120; ## 2 minutes.<br>
}<br>
<br>
disk {<br>
on-io-error detach;<br>
}<br>
<br>
net {<br>
# timeout 60;<br>
# connect-int 10;<br>
# ping-int 10;<br>
# max-buffers 2048;<br>
# max-epoch-size 2048;<br>
}<br>
<br>
syncer {<br>
after "cluster_metadata";<br>
}<br>
<br>
on <a href="http://storage3-a.wor.net" target="_blank">storage3-a.wor.net</a> {<br>
device /dev/drbd1;<br>
disk /dev/sda5;<br>
address <a href="http://192.168.93.3:7789" target="_blank">192.168.93.3:7789</a>;<br>
meta-disk internal;<br>
}<br>
<br>
on <a href="http://storage3-b.wor.net" target="_blank">storage3-b.wor.net</a> {<br>
device /dev/drbd1;<br>
disk /dev/sda5;<br>
address <a href="http://192.168.93.4:7789" target="_blank">192.168.93.4:7789</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
<br>
resource vg0drbd2 {<br>
protocol C;<br>
startup {<br>
wfc-timeout 0; ## Infinite!<br>
degr-wfc-timeout 120; ## 2 minutes.<br>
}<br>
<br>
disk {<br>
on-io-error detach;<br>
}<br>
<br>
syncer {<br>
after "vg0drbd";<br>
}<br>
<br>
on <a href="http://storage3-a.wor.net" target="_blank">storage3-a.wor.net</a> {<br>
device /dev/drbd2;<br>
disk /dev/sdb5;<br>
address <a href="http://192.168.93.3:7790" target="_blank">192.168.93.3:7790</a>;<br>
meta-disk internal;<br>
}<br>
<br>
on <a href="http://storage3-b.wor.net" target="_blank">storage3-b.wor.net</a> {<br>
device /dev/drbd2;<br>
disk /dev/sda6;<br>
address <a href="http://192.168.93.4:7790" target="_blank">192.168.93.4:7790</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
resource vg0drbd3 {<br>
protocol C;<br>
startup {<br>
wfc-timeout 0; ## Infinite!<br>
degr-wfc-timeout 120; ## 2 minutes.<br>
}<br>
<br>
disk {<br>
on-io-error detach;<br>
}<br>
<br>
syncer {<br>
after "vg0drbd2";<br>
}<br>
<br>
<br>
on <a href="http://storage3-a.wor.net" target="_blank">storage3-a.wor.net</a> {<br>
device /dev/drbd3;<br>
disk /dev/sdc5;<br>
address <a href="http://192.168.93.3:7791" target="_blank">192.168.93.3:7791</a>;<br>
meta-disk internal;<br>
}<br>
<br>
on <a href="http://storage3-b.wor.net" target="_blank">storage3-b.wor.net</a> {<br>
device /dev/drbd3;<br>
disk /dev/sda7;<br>
address <a href="http://192.168.93.4:7791" target="_blank">192.168.93.4:7791</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
resource vg0drbd4 {<br>
protocol C;<br>
startup {<br>
wfc-timeout 0; ## Infinite!<br>
degr-wfc-timeout 120; ## 2 minutes.<br>
}<br>
<br>
disk {<br>
on-io-error detach;<br>
}<br>
<br>
syncer {<br>
after "vg0drbd3";<br>
}<br>
<br>
<br>
on <a href="http://storage3-a.wor.net" target="_blank">storage3-a.wor.net</a> {<br>
device /dev/drbd4;<br>
disk /dev/sdd5;<br>
address <a href="http://192.168.93.3:7792" target="_blank">192.168.93.3:7792</a>;<br>
meta-disk internal;<br>
}<br>
<br>
on <a href="http://storage3-b.wor.net" target="_blank">storage3-b.wor.net</a> {<br>
device /dev/drbd4;<br>
disk /dev/sda8;<br>
address <a href="http://192.168.93.4:7792" target="_blank">192.168.93.4:7792</a>;<br>
meta-disk internal;<br>
}<br>
}<br>
<br>
<br>
Greetings from Germany,<br>
Andreas<br>
<br>
<br>
<br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
</blockquote></div><br>