Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
I think you can also mess with your al-extents setting to increase performance, but I think it only helps you get through bursts of size[al-extents * 4M] and has other repercussions later as well. (You'll resync that much after a primary crash). I think this mostly helps on systems with sub-optimal disks or controller cards. We had to set with high with our 3ware cards, but with our Areca's we've been just fine.. The problem with our 3ware cards was that when we had x-fer's > al-extents * 4M the system would be fine until it had xfer'd that amount, or a little bit more, but then it would nearly deadlock after that limit was reached. Kinda like the sorta thing that could keep people from logging in effectively, or keeping their active connections.. Our system would go into disk IO deadlock for about 30 seconds -> a minute, then come out of it for about 10 seconds, then go right back into another deadlock. This would continue until shortly after our large transfer's were done.. Any transfer that was smaller than al-extents * 4M would never have this problem (unless run too closely to another large transfer). I'm 100% sure that our problem was related to a problem with either our disks or our raid controller (3ware 85xx), and not a problem with drbd. The 3ware cards seemed to be OK if you only had a couple or a few disks, but when you kept adding more drives the problem would get progressively worse when running in a raid 5 configuration. We had either 8 or 12 drives on our system and it ugly painful. On Wed, 2006-06-21 at 10:47 -0700, Tim Johnson wrote: > Tim Johnson > Senior Software Engineer > Vision Solutions, Inc. > > 17911 Von Karman Ave, 5th Floor > Irvine, CA 92614 > UNITED STATES > > Tel: +1 (949) 253-6528 > Fax: +1 (949) 225-0287 > Email: tjohnson at visionsolutions.com > <http://www.visionsolutions.com/> > Disclaimer - 6/21/2006 > The contents of this e-mail (and any attachments) are confidential, may be privileged, and may contain copyright material of Vision Solutions, Inc. or third parties. You may only reproduce or distribute the material if you are expressly authorized by Vision Solutions to do so. If you are not the intended recipient, any use, disclosure or copying of this e-mail (and any attachments) is unauthorized. If you have received this e-mail in error, please immediately delete it and any copies of it from your system and notify us via e-mail at helpdesk at visionsolutions.com > -----Original Message----- > From: drbd-user-bounces at lists.linbit.com > [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Tim Johnson > Sent: Wednesday, June 21, 2006 10:47 AM > To: drbd-user at lists.linbit.com > Subject: RE: [DRBD-user] drbd and lvm understanding question > > Hi guys, > > We've got a bit of a problem at a customer site and I was wondering if > anybody had any suggestions. With drbd up and running on both the > primary and backup, massive amounts of data were copied over to the > relevant mount points on the primary. Apparently this slowed the > machine down so much (haven't yet been able to get from them if it was > the CPU usage or memory) that users were getting kicked off. When drbd > was taken down on the backup, then everything was o.k. They started > with drbd version 0.7.13. I perused the archives of this mailing list > and found something which suggested that this was a problem fixed after > 0.7.13, so they upgraded to 0.7.19 and are still having the problem. > Parameters we've thought might be appropriate in the drbd.conf file are > protocol (using protocol A: I'm sure this is fine), sndbuf-size > (warnings using large values like 1M), max-buffers (this looks promising > to me), max-epoch-size, and maybe rate. I'm a bit nervous about > changing anything, so does anybody have some good ideas? > > Appropriate environmental information such as proc/drbd and system info > is below: > > Thanks, > Tim > > ----------------------------------------------------------------- > Output from /proc/drbd > smb-mtl01:~ # cat /proc/drbd > version: 0.7.19 (api:78/proto:74) > SVN Revision: 2212 build by root at smb-mtl01, 2006-06-13 10:56:44 > 0: cs:Connected st:Primary/Secondary ld:Consistent > ns:0 nr:0 dw:2112 dr:209304 al:522 bm:0 lo:0 pe:0 ua:0 ap:0 > 1: cs:Connected st:Primary/Secondary ld:Consistent > ns:32 nr:0 dw:352 dr:86220 al:3 bm:2 lo:0 pe:0 ua:0 ap:0 > 2: cs:Connected st:Primary/Secondary ld:Consistent > ns:32 nr:0 dw:1632 dr:123328 al:393 bm:3 lo:0 pe:0 ua:0 ap:0 > > > smb-sjo01:~ # cat /proc/drbd > version: 0.7.19 (api:78/proto:74) > SVN Revision: 2212 build by root at smb-sjo01, 2006-06-12 14:55:54 > 0: cs:Connected st:Secondary/Primary ld:Consistent > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 > 1: cs:Connected st:Secondary/Primary ld:Consistent > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 > 2: cs:Connected st:Secondary/Primary ld:Consistent > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 > ------------------------------------------------------------------ > > Machine 1: (primary, I believe) > Link speed: 10 MBps (T1 line) > 1 NIC 100 MB/s. > SuSe 9 > Hardware Configuration: > Manufactured by IBM (iSeries) running on top of AS/400 810. > Has 1 CPU at 1GHz. > Memory is 1024 MB. > 200 GB disk > > > Machine 2: > Also SuSE 9. > Same IBM running on iSeries. > 1 CPU at 1 GHz. > 512 MB memory. > 150 GB disk space. > 1 NIC at 10 MB/s > > Replicating about 100 GB of data. > > > Tim Johnson > Senior Software Engineer > Vision Solutions, Inc. > > 17911 Von Karman Ave, 5th Floor > Irvine, CA 92614 > UNITED STATES > > Tel: +1 (949) 253-6528 > Fax: +1 (949) 225-0287 > Email: tjohnson at visionsolutions.com > <http://www.visionsolutions.com/> > Disclaimer - 6/21/2006 > The contents of this e-mail (and any attachments) are confidential, may > be privileged, and may contain copyright material of Vision Solutions, > Inc. or third parties. You may only reproduce or distribute the material > if you are expressly authorized by Vision Solutions to do so. If you are > not the intended recipient, any use, disclosure or copying of this > e-mail (and any attachments) is unauthorized. If you have received this > e-mail in error, please immediately delete it and any copies of it from > your system and notify us via e-mail at helpdesk at visionsolutions.com > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user