[DRBD-user] drbd 0.7.4 half the speed of 0.6.13

[Lars Ellenberg]

On Fri, Sep 17, 2004 at 04:08:52PM -0700, Josh McAllister wrote:
> I've tried both on the same hosts, same drives (15K SCSI Ultra160
> drives)  nothing changing except DRBD vers. (and conf. of course).
> 
> version: 0.6.13 (api:64/proto:62)
> 
> 0: cs:SyncingAll st:Primary/Secondary ns:76445216 nr:0 dw:0 dr:76445216
> pe:157 ua:0
>         [=>..................] sync'ed:  6.7% (65350/70001)M
>         finish: 0:16:09h speed: 73,575 (73,281) K/sec
> 
> 
> version: 0.7.4 (api:76/proto:74)
> SVN Revision: 1537M build by root at cluster3-1, 2004-09-17 14:58:25
>  0: cs:SyncSource st:Secondary/Secondary ld:Consistent
>     ns:855892 nr:0 dw:0 dr:856000 al:0 bm:52 lo:27 pe:0 ua:27 ap:0
>         [>...................] sync'ed:  1.3% (69043/69879)M
>         finish: 0:36:49 speed: 31,896 (27,608) K/sec
> 
> I've tried a plethora of different max-buffers / max-epoch values,
> starting with 2048/2048, and ending up at 256/256 as this improved
> performance from about 20MB/sec to about 28MB/sec. Still no where near
> the 73MB/sec I can reliably obtain using 0.60.13
> 
> Now, I have noted that load avg on 0.7.4 is quite a bit lower than it is
> under 0.6.13. But unless I'm mistaken sync-nice is deprecated. I'd
> gladly give up some cpu cycles to increase throughput. Is there another
> way to do this in 0.7.4?
> 
> Please help! I'd like to be able to take advantage of the added benefits
> of 0.7.x, but this is for a DB server that is already I/O constrained...
> I can't afford to take this kind of performance hit.
> 
> Thanks,
> 
> Josh
> 
> Below are the resource config sections for each version

thank you for this report.

but please note that with 0.6.x, you sometimes need to sync the full
device, with 0.7.x you always only sync those parts of the disk that
actually differ.

so the typicall resync time will be shorter with 0.7. anyways!
and keep in mind that your application on top of drbd wants to
have _some_ bandwidth, too. actually the sync-rate parameter is to
throttle resync: we would rather "reserve" a certain amount of
application io bandwith.

but in fact, we still have an arbitrary throttle in the current
resync code.  would you please verify whether this patchlet helps
you to see better resync performance with 0.7?

	Lars Ellenberg


===================================================================
--- drbd_worker.c	(revision 1539)
+++ drbd_worker.c	(working copy)
@@ -464,11 +464,10 @@
 
         number = SLEEP_TIME*mdev->sync_conf.rate / ((BM_BLOCK_SIZE/1024)*HZ);
 
-        if(number > 1000) number=1000;  // Remove later
-	if (atomic_read(&mdev->rs_pending_cnt)>1200) {
-		// INFO("pending cnt high -- throttling resync.\n");
+	if (atomic_read(&mdev->rs_pending_cnt)>number) {
 		goto requeue;
 	}
+	number -= atomic_read(&mdev->rs_pending_cnt);
 
 	for(i=0;i<number;i++) {