Poor performance solved WAS: Re: [DRBD-user] 2.6.24-rc1 oops

Matteo Tescione matteo at rmnet.it
Sat Nov 10 02:10:39 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Leroy, drbd folks,

After a while of collecting and profiling my systems i discovered that our
poor performance problems were not related of processor, raid controller or
kernel configuration.
It's all about raid5/6 level configurations that is absolutely unreliable.

In details reading and writing directly to the raid disk, with an ext3
partition for example, gives more than 200 mbyte/sec with await and svc time
of approx 1.2 ms versus reading and writing with drbd (either syncing or
live i/o) gives extremely poor performance (i got 30-40 mbyte/sec with 3ware
9650-8p configured in raid6 and i have 20 mbytes/sec with Compaq Smart Array
p400 configured in raid5 mode.
In both raid5/6 configurations, i/o seems stucked as i've reported in my
previous post with very high iowait values (await up to 5000 ms, svctime up
to 1000 ms)

Switching to raid0 / raid10 configurations, totally solved our problems.
Sync its up to 200mbyte/sec, live i/o seems good as well but i'm still
testing the vmware infrastructure...

If anyone has any idea of what's going on....

So long and thank for all the fish

#Matteo Tescione
#RMnet srl

Il 7-11-2007 16:26, "Leroy van Logchem" <leroy.vanlogchem at wldelft.nl> ha

> Matteo wrote:
>> Hi Lars and thank for your quick answer, i had a little look at the kernel
>> Changelog and i saw they changed sg_set_page, suggestin use of
>> sg_assign_page.I went to drbd/drbd_worker.c changin sg_set_page to
>> sg_assign_page and it
>> compiles without problems. So i solved the oops usin the rc2...
>> BUT, even switching to new kernel my problems remains the same...
>> I have tested latest drbd/kernel with CORE2/QUAD processors and i'm in deep
>> troubles, syncer get stuck at 20mb/s with incredible high await/iowait times:
> You might want to instrument the kernel and use oprofile.
> URL         : http://oprofile.sf.net
> Summary     : System wide profiler
> Description :
> Profiling runs transparently during the background, and profile data
> can be collected at any time. oprofile makes use of the hardware performance
> counters provided on Intel P6, and AMD Athlon family processors, and can use
> the RTC for profiling on other x86 processor types.
> Enable both Kernel profiling and debugging support. Oprofile needs the
> uncompressed vmlinux.
> I don't know your distro but you can find some setup/use tips at:
> http://www.linuxtopia.org/online_books/centos5/centos5_administration_guide/ce
> ntos5_s1-oprofile-configuring.html
> Compare short benchmarks with cold caches using the P4 vs CORE2 with the
> same kernel setup. Include /bin/sync after the benchmark.
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user

More information about the drbd-user mailing list