Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi Pascal
Thank you.
I think the speed difference comes from how the ext4 filesystems were
created... On /dev/sda, /home, /opt, /var, /tmp, /usr are on LVM but on
/dev/sdb there's no LVM. It seems that at creation time the ext4 partitions
on LVM may have been better optimized. However I don't know how to get the
parameters of an existing ext4 partition.
Having said that I modify /etc/fstab to have:
/dev/drbd/by-res/mysql /var/lib/mysql ext4
rw,suid,dev,exec,noauto,nouser,async,barrier=0,data=writeback 0 0
instead of
/dev/drbd/by-res/mysql /var/lib/mysql ext4
rw,suid,dev,exec,noauto,nouser,async 0 0
and I got a 200MB/s instant boost making writing to /home (non-DRBD) or
/var/lib/mysql (DRBD) with secondary node off more or less the same with a
troughput of ~ 420MB/s
Kind regards,
Fred
On Wed, Feb 1, 2012 at 6:00 PM, Pascal BERTON (EURIALYS) <
pascal.berton at eurialys.fr> wrote:
> Frederic,****
>
> ** **
>
> Paravirtual SCSI is supposed to be fully efficient passed a given level of
> IOPS, under that level of activity an LSI SAS adapter used to be reputed
> being more performant (It was in v4, don’t have any update regarding v5, I
> suspect it’s still the case though). However, since both disks are plugged
> onto the same adapter and you issue the same test command for both disks,
> this can’t explain what you’re seeing… Therefore it seems like your problem
> is effectively whithin your VM, and not “around”. Let’s keep searching,
> then… J****
>
> ** **
>
> Best regards,****
>
> ** **
>
> Pascal.****
>
> ** **
>
> *De :* Frederic DeMarcy [mailto:fred.demarcy.ml at gmail.com]
> *Envoyé :* mercredi 1 février 2012 17:20
> *À :* Pascal BERTON (EURIALYS)
> *Cc :* drbd-user at lists.linbit.com
>
> *Objet :* Re: [DRBD-user] Slower disk throughput on DRBD partition****
>
> ** **
>
> Hi Pascal
>
> 1) Both vdisks for /dev/sda and /dev/sdb are on the same datastore which
> is made of the entire RAID5 array capacity (7xHDs + 1 spare). Minus space
> used by the ESXi installation.
> 2) HD1 (/dev/sda) is SCSI (0:1) and HD2 (/dev/sdb) is SCSI (0:2). Both
> initialized with Thick Provisioning Eager Zeroed. The SCSI controller type
> is paravirtual.
>
> Fred****
>
> On Wed, Feb 1, 2012 at 2:13 PM, Pascal BERTON (EURIALYS) <
> pascal.berton at eurialys.fr> wrote:****
>
> Frederic,
>
> Let's take care of the virtualisation layer wich might induce significant
> side effects
> Are sda and sdb :
> 1) vdisk files located on the same datastore ?
> 2) vdisks plugged on the same virtual SCSI interface ? What type of SCSI
> interface ?
>
> Best regards,
>
> Pascal.
>
> -----Message d'origine-----
> De : drbd-user-bounces at lists.linbit.com
> [mailto:drbd-user-bounces at lists.linbit.com] De la part de Frederic DeMarcy
> Envoyé : mercredi 1 février 2012 13:05
> À : drbd-user at lists.linbit.com
> Objet : Re: [DRBD-user] Slower disk throughput on DRBD partition****
>
>
> Hi
>
> Note 1:
> Scientific Linux 6.1 with kernel 2.6.32-220.4.1.el6.x86_64
> DRBD 8.4.1 compiled from source
>
> Note 2:
> server1 and server2 are 2 VMware VMs on top of ESXi 5. However they reside
> on different physical 2U servers.
> The specs for the 2U servers are identical:
> - HP DL380 G7 (2U)
> - 2 x Six Core Intel Xeon X5680 (3.33GHz)
> - 24GB RAM
> - 8 x 146 GB SAS HD's (7xRAID5 + 1s)
> - Smart Array P410i with 512MB BBWC
>
> Note 3:
> I've tested the network throughput with iperf which yields close to 1Gb/s
> [root at server1 ~]# iperf -c 192.168.111.11 -f g
> ------------------------------------------------------------
> Client connecting to 192.168.111.11, TCP port 5001
> TCP window size: 0.00 GByte (default)
> ------------------------------------------------------------
> [ 3] local 192.168.111.10 port 54330 connected with 192.168.111.11 port
> 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0-10.0 sec 1.10 GBytes 0.94 Gbits/sec
>
> [root at server2 ~]# iperf -s -f g
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 0.00 GByte (default)
> ------------------------------------------------------------
> [ 4] local 192.168.111.11 port 5001 connected with 192.168.111.10 port
> 54330
> [ ID] Interval Transfer Bandwidth
> [ 4] 0.0-10.0 sec 1.10 GBytes 0.94 Gbits/sec
>
> Scp'ing a large file from server1 to server2 yields ~ 57MB/s but I guess
> it's due to the encryption overhead.
>
> Note 4:
> MySQL was not running.
>
>
>
> Base DRBD config:
> resource mysql {
> startup {
> wfc-timeout 3;
> degr-wfc-timeout 2;
> outdated-wfc-timeout 1;
> }
> net {
> protocol C;
> verify-alg sha1;
> csums-alg sha1;
> data-integrity-alg sha1;
> cram-hmac-alg sha1;
> shared-secret "MySecret123";
> }
> on server1 {
> device /dev/drbd0;
> disk /dev/sdb;
> address 192.168.111.10:7789;
> meta-disk internal;
> }
> on server2 {
> device /dev/drbd0;
> disk /dev/sdb;
> address 192.168.111.11:7789;
> meta-disk internal;
> }
> }
>
>
> After any change in the /etc/drbd.d/mysql.res file I issued a "drbdadm
> adjust mysql" on both nodes.
>
> Test #1
> DRBD partition on primary (secondary node disabled)
> Using Base DRBD config
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Throughput ~ 420MB/s
>
> Test #2
> DRBD partition on primary (secondary node enabled)
> Using Base DRBD config
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Throughput ~ 61MB/s
>
> Test #3
> DRBD partition on primary (secondary node enabled)
> Using Base DRBD config with:
> Protocol B;
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Throughput ~ 68MB/s
>
> Test #4
> DRBD partition on primary (secondary node enabled)
> Using Base DRBD config with:
> Protocol A;
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Throughput ~ 94MB/s
>
> Test #5
> DRBD partition on primary (secondary node enabled)
> Using Base DRBD config with:
> disk {
> disk-barrier no;
> disk-flushes no;
> md-flushes no;
> }
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Disk throughput ~ 62MB/s
>
> No difference from Test #2 really. Also cat /proc/drbd still shows wo:b in
> both cases so I'm not even sure
> these disk {..} parameters have been taken into account...
>
> Test #6
> DRBD partition on primary (secondary node enabled)
> Using Base DRBD config with:
> Protocol B;
> disk {
> disk-barrier no;
> disk-flushes no;
> md-flushes no;
> }
> # dd if=/dev/zero of=/var/lib/mysql/TMP/disk-test.xxx bs=1M count=4096
> oflag=direct
> Disk throughput ~ 68MB/s
>
> No difference from Test #3 really. Also cat /proc/drbd still shows wo:b in
> both cases so I'm not even sure
> these disk {..} parameters have been taken into account...
>
>
> What else can I try?
> Is it worth trying DRBD 8.3.x?
>
> Thx.
>
> Fred
>
>
>
>
>
>
> On 1 Feb 2012, at 08:35, James Harper wrote:
>
> >> Hi
> >>
> >> I've configured DRBD with a view to use it with MySQL (and later on
> >> Pacemaker + Corosync) in a 2 nodes primary/secondary
> >> (master/slave) setup.
> >>
> >> ...
> >>
> >> No replication over the 1Gb/s crossover cable is taking place since the
> >> secondary node is down yet there's x2 lower disk performance.
> >>
> >> I've tried to add:
> >> disk {
> >> disk-barrier no;
> >> disk-flushes no;
> >> md-flushes no;
> >> }
> >> to the config but it didn't seem to change anything.
> >>
> >> Am I missing something here?
> >> On another note is 8.4.1 the right version to use?
> >>
> >
> > If you can do it just for testing, try changing to protocol B with one
> primary and one secondary and see how that impacts your performance, both
> with barrier/flushes on and off. I'm not sure if it will help but if
> protocol B makes things faster then it might hint as to where to start
> looking...
> >
> > James****
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user****
>
> ** **
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20120202/9d877bfa/attachment.htm>