Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Igor, what kernel r u running? Can I have a look of your config.gz? And also what io scheduler? I'm facing problems where linux 2.6.23 with drbd 8.2.0 is having 200MB/sec with a couple of P4 with 3ware storage and e1000 bonded, and INCREDIBLE, switching the same kernel/os to our Xeon 53 series, with intel server board s5000psl dropped performance to about 40-50 MB/sec, usin the same 3ware array, e1000 bonds etc... Obviously there's nothing wrong in drbd.conf or other stuff, iperf gave me 1,50 gbit without problems, disktest from ltp gave me up to 300 mb/sec to the physical storage and so on.... It seems that's somethin wrong in latest kernel, i attach my .config .... Any ideas? -- #Matteo Tescione #RMnet srl Il 24-10-2007 18:03, "Igor Neves" <igor at 3gnt.net> ha scritto: > Hi, > > Here is more information about my hardware: > > 08:0e.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS > > Driver: megaraid_sas > > Partitions on megaraid controller: > > major minor #blocks name > > 8 0 35155968 sda > 8 1 104391 sda1 > 8 2 35045797 sda2 > 8 16 570310656 sdb > 8 17 570307468 sdb1 > 147 0 570290024 drbd0 > > On Wed, 24 Oct 2007 09:04:11 -0500 > Todd Denniston <Todd.Denniston at ssa.crane.navy.mil> wrote: > >> Igor Neves wrote, On 10/24/2007 04:21 AM: >>> Hi, >>> >>> On Tue, 23 Oct 2007 14:52:33 -0400 (EDT) >>> Nate Seif <seifn at rstechnicalservices.com> wrote: >>> >>>> Identify where the bottleneck is to [possibly] get faster sync >>>> speeds: (1) Check to make sure your kernel has proper [driver] >>>> support for your chipset. Run 'lspci' to get info about what >>>> chipset you have if you don't already know. >>> >>> I have support, and i have full support for the device. >>> >>>> (2) Make sure you have DMA enabled for your hard drives. Research >>>> first, then run 'hdparm' to test write speeds to see if hard drives >>>> are the bottleneck. >>> >>> DMA in my case, have nothing to do with it, it's SAS device. >>> >>>> (3) Check your network: can you get faster transfer speeds with >>>> other app's? Make sure you've got the proper drivers for your >>>> NIC's... >>> >>> I get much faster transfer on my NIC's, yes, i have tested with >>> iperf. >>> >> >> Have you seen the speed tricks from the list? >> >> http://www.google.com/search?hl=en&q=speed+gigabit.ethernet+site%3Alists.linb >> it.com%2Fpipermail%2Fdrbd-user%2F&btnG=Google+Search >> >> Oh, and this next guy _usually_ knows what he is talking about, >> though he was using an older drbd and you may need to figure out how >> some of the settings map to current versions: :) >> http://lists.linbit.com/pipermail/drbd-user/2004-November/002241.html >> >> And another recent speed thread: >> http://lists.linbit.com/pipermail/drbd-user/2007-June/007010.html >> http://lists.linbit.com/pipermail/drbd-user/2007-June/007033.html >> >> >> Also, are you running multiple DRBD's? >> If so do you have them all on one spindle or Array? >> Are you using Sync groups? >> >> What protocol (A,B,C) are you using (should not matter with initial >> sync, just being pedantic)? > > All this questions, in the attach drbd.conf, :). > >> >> a few more comments below >> >>>> Good luck. >>> >>> Thanks. >>> >>>> Nate >>>> >>>> >>>> On Tue, 23 Oct 2007, Igor Neves wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm having problems with sync, using drbd 8.0.4, in centos 5.0. >>>>> >>>>> When i have to do one full sync, of the disc, it takes me 12 Hours >>>>> (500G). >> >> so you are saying ... >> echo -en "scale=2\n(500*1024*1024)/(12*60*60)\n"|bc >> .. you are seeing ~12MBytes/second going through? > > Yes, exactly 12MBytes, which is ridiculous for gigabit ethernet and this > hardware, and it's exactly how it takes, about 10~12 hours to finish > the 500G. > >> >> Looks suspiciously like the 100Mbit network max. >> >> try giving us `cat /proc/drbd` output so we are not making up numbers >> from yours. :) >> >>>>> >>>>> The 2 nodes are connected between Gigabit Ethernet, my question >>>>> it's, what do i have to do, to make it faster? This time is >>>>> insane! :) >>>>> >>>>> I know my disk does not write at Gigabit ethernet capacity, but >>>>> indeed, lets say it writes at least at 50MBytes/sec. >>>>> >>>>> If we write data, at 50MBytes/sec it should take theoretically >>>>> about 1.66 hours, its about 1h and 40 minutes or so. >>>>> >>>>> Of course the disk writes faster than that, at least in this case >>>>> i have one SAS controller with 15k rpm disk's. >>>>> >>>>> I have in drbd.conf configured 'rate 100M'. >>>>> >>>>> I have drbd over the disk, and ext3 over it, but i think this is >>>>> transparent for drbd, this should not take the drbd to slow. >>>>> >>>>> When it is doing the full sync, i see in '/proc/drbd' and it says >>>>> it's copying at about 20K/sec, and since it is in bits, it is >>>>> pretty slow. >>>>> >> >> IIRC, the data in /proc/drbd is in KBytes, so you are seeing >> 20KBytes/second (or if you did a little math wrong for us, >> 20MBytes/second). The following is indicating ~15MBytes/seconds: >> finish: 4:20:22h speed: 15,290 (15,374) K/sec > > This is not easy since i dont want to sync for 12 hours again! :) > > But yes, its that ~12-15Mb i get. > >> >> >>>>> This is bug? Or I am the bug? :) Since I'm new to drbd, i think >>>>> the second option will be more likely then the first one! :D >>>>> >>>>> Gigabit Ethernet = 125MBytes/Sec >>>>> >>>>> Thanks for all the help. >> >> > > Thanks for all your help guys. > --------- > Igor Neves <igor.neves at 3gnt.net> > 3GNTW - Tecnologias de Informação, Lda > > sip igor at 3gnt.net jid igor at jabber.3gnt.org > icq 249075444 tlm 00351914503611 > > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user