<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
On 09/01/2010 06:29 PM, Robert Verspuy wrote:
<blockquote cite="mid:4C7E7F68.9010305@exa-omicron.nl" type="cite">*Latency:
(changed to write 1000 times 4k)*
<br>
<br>
DRBD Connected:
<br>
4096000 bytes (4.1 MB) copied, 24.7744 seconds, 165 kB/s
<br>
<br>
DRBD Disconneted:
<br>
4096000 bytes (4.1 MB) copied, 0.198809 seconds, 20.6 MB/s
<br>
<br>
</blockquote>
I've moved the meta-data to a ramdisk on both servers, but the
latency is still the same.<br>
Also tried the deadline scheduler, enable / disable <br>
<br>
One other thing I saw was the amount of interrupts on de secondary
server.<br>
The 8 SATA disks are connected to 2 sata controllers.<br>
<br>
During the latency test the amount of interrupts<br>
on the primary server on the sata_mv card is around 1170.<br>
on the secondary server on the sata_mv card is around 1804.<br>
<br>
On the primary server on the ahci card is around 376<br>
On the secondary server on the ahci card is around 4308<br>
<br>
So for the ahci driver I have about 10 times more interrupts on the
secondary server, then on the primary server.<br>
<br>
I just tried to add the options no-disk-barrier, but no difference.<br>
When added no-disk-flushes, I suddenly got good performance numbers:<br>
<br>
So the extra latency is somewhere in the flushing part by drbd on
the md device.<br>
But why is the flushing on the secondary much slower, compared to
flushing on the primary?<br>
I also tested this with running db01 as primary, disconnected db02,<br>
and running the latency test on /dev/drbd0.<br>
<br>
I assume (because now I'm still writing to the drdb device but
without active secondary), that drbd is still running the flushes on
/dev/md2?<br>
<br>
Also tested this while making db02 primary, and db01 secondary, to
see if there would be any raid problems on db02,<br>
but I get exactly the same results.<br>
<br>
When running the the latency test on the primary without secondary
attached<br>
[root@db02 drbd]# cat /proc/drbd <br>
version: 8.3.8 (api:88/proto:86-94)<br>
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by
<a class="moz-txt-link-abbreviated" href="mailto:mockbuild@builder10.centos.org">mockbuild@builder10.centos.org</a>, 2010-06-04 08:04:09<br>
0: cs:WFConnection ro:Primary/Unknown ds:UpToDate/DUnknown B r----<br>
ns:4000 nr:4000 dw:12000 dr:0 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1
wo:f oos:4000<br>
[root@db02 drbd]# dd if=/dev/zero of=/dev/drbd0 bs=4k count=1000
oflag=direct<br>
1000+0 records in<br>
1000+0 records out<br>
4096000 bytes (4.1 MB) copied, 0.205476 seconds, 19.9 MB/s<br>
<br>
With secondary arracted:<br>
[root@db02 drbd]# cat /proc/drbd <br>
version: 8.3.8 (api:88/proto:86-94)<br>
GIT-hash: d78846e52224fd00562f7c225bcc25b2d422321d build by
<a class="moz-txt-link-abbreviated" href="mailto:mockbuild@builder10.centos.org">mockbuild@builder10.centos.org</a>, 2010-06-04 08:04:09<br>
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate B r----<br>
ns:8000 nr:4000 dw:12000 dr:4000 al:1 bm:2 lo:0 pe:0 ua:0 ap:0
ep:1 wo:f oos:0<br>
[root@db02 drbd]# dd if=/dev/zero of=/dev/drbd0 bs=4k count=1000
oflag=direct<br>
1000+0 records in<br>
1000+0 records out<br>
4096000 bytes (4.1 MB) copied, 27.4238 seconds, 149 kB/s<br>
<br>
But when using Protocol C, and no-disk-barrier and no-disk-flushes,
I get:<br>
[root@db01 drbd-test]# dd if=/dev/zero of=/dev/drbd0 bs=4k
count=1000 oflag=direct<br>
1000+0 records in<br>
1000+0 records out<br>
4096000 bytes (4.1 MB) copied, 0.504855 seconds, 8.1 MB/s<br>
<br>
I also see according to dstat, that the 4.1 MBytes are saved to disk
on the secondary within 2 seconds after the dd command.<br>
<br>
The documentaton states that you must not use no-disk-barrier and
no-disk-flushes when you don't have a raid controller with battery
backup.<br>
But is this really gonna give me problems?<br>
Both servers have 2 power supplies, attached to 2 power feeds in a
data centre.<br>
So the change both servers will loose power at the same time is
already very minimal.<br>
And even then both server loose power, the postgresql database is
ACID, so there will always be a consistent data.<br>
Then I may have lost a few database updates / inserts, but we're not
going to run a database for banking or a nuclear power reactor :)<br>
<br>
So I think I will setup DRBD with Protocol C and no-disk-barrier and
no-disk-flushes.<br>
<br>
<div class="moz-signature">-- <br>
<b>Exa-Omicron</b><br>
Patroonsweg 10<br>
3892 DB Zeewolde<br>
Tel.: 088-OMICRON (66 427 66)<br>
<a class="moz-txt-link-freetext" href="http://www.exa-omicron.nl">http://www.exa-omicron.nl</a></div>
</body>
</html>