[DRBD-user] performance help with new servers

[Kushnir, Michael (NIH/NLM/LHC) [C]]

Hi Morey,

It really depends on your RAID config and your network card. I am using Dell PE-C 2100s with LSI's  9260-8i RAID card with 1GB RAM on board. I have write-back caching enabled and read-ahead caching as well. My RAID10 sets on each server is made up of 10 x 1TB Seagate Constellation 3Gb/s SATA drives with 1MB stripe size. My 10GbE cards are Myricom SFP+ cards. On the initial sync I was able to squeeze about 350MB/s transfer speeds between servers. RAIDs 5 and 6 will give you worse performance. RAID 10, 50, and 60 should do much better. Newer LSI cards also have the option of adding SSDs as read and/or write cache for your RAID array and various SSD protection features to make them last. You will need the LSI GUI for Linux to find these features easily. Also, I've heard of notoriously poor performance on 1st gen Intel cards on Linux.

My setup is still pre-production so my drbd.conf file is not finalized and has some non-advisable entries, but please take a look:

global {
# minor-count 64;
# dialog-refresh 5; # 5 seconds
# disable-ip-verification;
usage-count no;
}

common {

syncer {
        rate 100M; # My bottleneck is RAID array so this is set so I can still use storage during sync.
        #al-extents 257;
        }

handlers {
        #fence-peer "/usr/lib64/heartbeat/drbd-peer-outdater -t 5";
        #pri-on-incon-degr "echo O > /proc/sysrq-trigger ; halt -f";
        #pri-lost-after-sb "echo O > /proc/sysrq-trigger ; halt -f";
        #local-io-error "echo O > /proc/sysrq-trigger ; halt -f";
        }


disk {
        fencing resource-only;
        no-disk-barrier;
        #use-bmbv;
        on-io-error detach;
        no-disk-flushes;
        }

net {
        # data-integrity-alg md5;
        allow-two-primaries;
        after-sb-0pri discard-older-primary;
        after-sb-1pri discard-secondary;
        #after-sb-0pri disconnect;
        #after-sb-1pri disconnect;
        after-sb-2pri disconnect;
        #rr-conflict disconnect;


}

startup {
        wfc-timeout 5;
        degr-wfc-timeout 5
        become-primary-on both;
        }
}

resource drbd0 {
protocol C;

on **** {
  device /dev/drbd0;
  disk /dev/sdb1;
  address ****;
  meta-disk internal;
}

on ****{
  device /dev/drbd0;
  disk /dev/sdb1;
  address ****;
  meta-disk internal;
  }
}

Best,
Michael


From: Roof, Morey R. [mailto:MRoof at admin.nmt.edu]
Sent: Thursday, September 29, 2011 1:48 PM
To: drbd-user at lists.linbit.com
Subject: [DRBD-user] performance help with new servers

Hi Everyone,

I got some new servers recently and they have the LSI Megaraid based cards (with battery backup units) with 10K SAS drives and 10Gbe cards.  At the moment the performance is quite less than I would like with them, about 90MB/s with DRBD protocol C.  My older HP servers do much better than this so if anyone has some similar servers and get a lot better performance could you share your drbd.conf file so I can help figure out what I'm missing?

Thanks,
Morey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20110929/ac2c2c28/attachment.htm>