[DRBD-user] DRBD Crashing/Stalling on Sync

[Adam Taylor]

:) Good stuff.  I have seen the same issue on a few installs were I have had
to set sync rate to about the same which seems to resolve the initial sync
issue.  What version were you using?  

Cheers

Adam 

-----Original Message-----
From: tpawlowski.itx at gmail.com [mailto:tpawlowski.itx at gmail.com] On Behalf
Of Tom Pawlowski
Sent: Wednesday, September 30, 2009 8:00 AM
To: Adam Taylor
Cc: drbd-user at lists.linbit.com
Subject: Re: [DRBD-user] DRBD Crashing/Stalling on Sync

Hey Adam,

The VPS gets hammered pretty hard, so I've had to throttle the sync rate
down pretty far. I believe when this happened, it was around 15-20MB/s on a
Gigebit crossover cable.

That being said, I managed to successfully run a sync last night without any
interruption or stalls at the same speeds. So, that's a relief. :)

More than likely, I guess the only solution would have been to update DRBD
if it persisted. I think from here on out I'll deploy newer versions than
what comes with Debian Lenny.

Thanks folks!

--T

On Tue, Sep 29, 2009 at 4:50 PM, Adam Taylor <adam.taylor at wml.co.nz> wrote:
> Hi Tom,
>
> Have you tried lowering your Sync Rate in drbd.conf?
>
> Thanks
>
> Adam
>
> -----Original Message-----
> From: drbd-user-bounces at lists.linbit.com
> [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Tom Pawlowski
> Sent: Tuesday, September 29, 2009 8:41 AM
> To: drbd-user at lists.linbit.com
> Subject: [DRBD-user] DRBD Crashing/Stalling on Sync
>
> Hi all,
>
> I've come across an issue on a clustered set up that I haven't been 
> able to find a resolution for. (It's a bit different than the other 
> stalled ticket as neither of the nodes are locking up on their own.
> They remain stable and are able to ping each other on both eth0/eth1.) 
> If anyone could shed some light on it, I'd very much appreciate it.
>
>
> *SETUP*
>
> Two hardware nodes, running a DRBD/LVM/Xen stack (in that order). One 
> node is using a RAID-1 3ware controller with two Western Digital Blue 
> 1.0TB drives, the other RAID-5 3ware controller with three WD Black RE 
> 500GB drives. Everything else is identical.
>
> (I'm attempting to sync the data to the RAID-5 node so I can failover 
> to that and replace RAID-1 on the other node--given that the 
> performance with DRBD on the latter is awful.)
>
> Controller information:
>
> node1
> 3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xda100000, IRQ:
> 16.
> 3w-9xxx: scsi0: Firmware FE9X 4.06.00.004, BIOS BE9X 4.05.00.015, Ports:
4.
>
> node2
> 3w-9xxx: scsi0: Found a 3ware 9000 Storage Controller at 0xda100000, IRQ:
> 16.
> 3w-9xxx: scsi0: Firmware FE9X 4.06.00.004, BIOS BE9X 4.05.00.015, Ports:
2.
>
> Software Version information for both nodes are identical:
>
> Distro: Debian 5.0.3
> Kernel: Linux 2.6.26-2-xen-amd64 #1 SMP Thu Aug 20 00:36:34 UTC 2009
> x86_64 GNU/Linux
> drbdadm Version: Version: 8.0.14 (api:86) Xen Version: 3.0.3
>
>
> *PROBLEM*
>
> The sync will run along without any issues for a few hours, after 
> which /proc/drbd reports that it has stalled. The Xen instance running 
> on top stops responding entirely. There are a number of stack traces 
> in the system log which I have attached to the email.
>
> Has anyone come across something like this before? Updating DRBD is a 
> bit iffy, as our client is very downtime-adverse given that he's 
> paying for a high-availability setup. Don't know if that can be done 
> without reinitializing both resources.
>
> Thanks in advance for any help!
>
> Regards,
> Tom Pawlowski
>
>
>
> __________ NOD32 4468 (20090929) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>

__________ NOD32 4468 (20090929) Information __________

This message was checked by NOD32 antivirus system.
http://www.eset.com