Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi, Can you please tell me, why are you doing software raid? Of all my experience drbd goes really bad with software raid. I have done tests with Lsi, 3ware, Areca and software raid with and without lvm, with ext3 and with xfs and drbd on top of it. I have clients with all the cards i mention, and none with software raid. When you get high load (drbd on top of it) and software raid working over it, it will get very low transfers and so much I/O wait... try to put virtual machines on top of that and you will be surprised. This is my opinion, if you have a big storage backend (your case) and depending on what you will want to get from the solution, use hardware raid with areca cards. Forget LSI, forget 3ware, they are garbage. Try to use pcie cards, they will perform a lot better then pci-x and with less problems. Maybe you have some problem there, and lsi's are not very fast, but so you can get the idea, i can get ~170MB/sec over drbd with one low end areca card with 4 stupid sata disks (7.5k rpm) in raid5. With your system, it should fly with one areca card. By the way i have a system with your LSI card with 6 SAS disks, and the best i can get of it was about ~150MB/sec. This examples it's just so you can get one idea of the other side of the world! :) Good luck for your problem, i dont know if i can help you more. Keep in mind, avoid software raid with drbd on top, they dont work well together for perfomance. Lars Täuber wrote: > Hi Igor, > > Igor Neves <igor at 3gnt.net> schrieb: > >> Hi, >> >> Are you using hardware raid under drbd device or software raid? >> >> Cheers >> >> > > it is software raid6 over 15 disks with one spare. The disks are in an SAS cage connected through > LSI Logic / Symbios Logic SAS1068 PCI-X Fusion-MPT SAS controller. > > # hdparm -t /dev/md4 > > /dev/md4: > Timing buffered disk reads: 1102 MB in 3.01 seconds = 366.57 MB/sec > Timing buffered disk reads: 1040 MB in 3.00 seconds = 346.63 MB/sec > Timing buffered disk reads: 1042 MB in 3.01 seconds = 346.62 MB/sec > Timing buffered disk reads: 1070 MB in 3.00 seconds = 356.16 MB/sec > Timing buffered disk reads: 1078 MB in 3.00 seconds = 358.83 MB/sec > Timing buffered disk reads: 1120 MB in 3.00 seconds = 373.25 MB/sec > > # cat /proc/mdstat > Personalities : [raid1] [raid0] [raid6] [raid5] [raid4] > md4 : active raid6 sdc sdr(S) sdq sdp sdo sdn sdm sdl sdk sdj sdi sdh sdg sdf sde sdd > 12697912448 blocks level 6, 64k chunk, algorithm 2 [15/15] [UUUUUUUUUUUUUUU] > > > Regards > Lars > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20090112/dba157a7/attachment.htm>