Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Thanks for responding,
FYI: I have ran stat command to get details of the files whose data is seen criss-crossing. I mean content of one file is seen in another. Snapshot enclosed at the end, when corruption occured.
Files which have an issue belong to same block, IO Block: 4096
Every corruption seen, content of /repl/firewall/sysconfig/iptables content is seen in /repl/snmpagent/data/snmpd.conf
How much is "few"?
Today After 12 failovers. Last run after 80 failovers similar corruption is seen.
What is the IO load?
Note exactly sure, When sigterm is received there are 2 processes which write config data to DRBD partition.
How do you trigger the failover?
using reboot command
DRBD version, kernel version, file system type?
DRBD-8.0.16, 2.6.14.7, EXT3-FS
Volatile caches involved?
NO
How often/when do you fsck?
Every time DRBD-GO-Primary script is called. Before mounting DRBD partition we invoke fsck -fy
File: `/repl/ipsec/ipsec_xml'
Size: 0 Blocks: 2 IO Block: 4096 regular empty file
Device: fe03h/65027d Inode: 6404 Links: 1
Access: (0640/-rw-r-----) Uid: ( 0/ root) Gid: ( 201/ admin)
Access: 2010-09-07 10:06:55.000000000 +0000
Modify: 2010-09-07 10:07:12.000000000 +0000
Change: 2010-09-07 10:07:12.000000000 +0000
File: `/repl/ipsec/psk.txt'
Size: 242 Blocks: 4 IO Block: 4096 regular file
Device: fe03h/65027d Inode: 6397 Links: 1
Access: (0600/-rw-------) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2006-08-03 17:03:19.000000000 +0000
Modify: 2010-09-07 10:07:12.000000000 +0000
Change: 2010-09-07 10:07:12.000000000 +0000
File: `/repl/ipsec/racoon.conf'
Size: 1793 Blocks: 6 IO Block: 4096 regular file
Device: fe03h/65027d Inode: 6391 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2010-09-07 10:02:49.000000000 +0000
Modify: 2010-09-07 10:07:12.000000000 +0000
Change: 2010-09-07 10:07:12.000000000 +0000
File: `/repl/ipsec/setkey.conf'
Size: 121 Blocks: 4 IO Block: 4096 regular file
Device: fe03h/65027d Inode: 6398 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2006-08-03 17:03:16.000000000 +0000
Modify: 2010-09-07 10:07:12.000000000 +0000
Change: 2010-09-07 10:07:12.000000000 +0000
File: `/repl/firewall/sysconfig/iptables'
Size: 1797 Blocks: 6 IO Block: 4096 regular file
Device: fe03h/65027d Inode: 14461 Links: 1
Access: (0600/-rw-------) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2010-09-07 10:02:51.000000000 +0000
Modify: 2010-09-07 10:07:13.000000000 +0000
Change: 2010-09-07 10:07:13.000000000 +0000
File: `/repl/snmpdagent/data/snmpd.conf'
Size: 683 Blocks: 4 IO Block: 4096 regular file
Device: fe03h/65027d Inode: 20744 Links: 1
Access: (0600/-rw-------) Uid: ( 0/ root) Gid: ( 601/usergroup)
Access: 2010-09-07 10:07:14.000000000 +0000
Modify: 2010-09-07 10:07:14.000000000 +0000
Change: 2010-09-07 10:07:14.000000000 +0000
Appreciate your help,
Lak.
> Date: Tue, 7 Sep 2010 12:16:59 +0200
> From: lars.ellenberg at linbit.com
> To: drbd-user at lists.linbit.com
> Subject: Re: [DRBD-user] File corruption in drbd partition
>
> On Tue, Sep 07, 2010 at 09:35:48AM +0000, putcha narayana wrote:
> >
> > Hi,
> >
> > We are running continuous failovers on a redundant setup (Active / Standby).
> > After few failovers we observe content of file x appears inside file y.
>
> How much is "few"?
> What is the IO load?
> How do you trigger the failover?
> DRBD version, kernel version, file system type?
> Volatile caches involved?
> How often/when do you fsck?
>
> > In one particular case we observed inode corruption, when fsck command is run on /repl partition.
> > Multiply-claimed block(s) in inode 28: 1233 1249 1251 1252
> > Multiply-claimed block(s) in inode 1183: 1251 1252
> > Multiply-claimed block(s) in inode 1184: 1233
> > Multiply-claimed block(s) in inode 1185: 1249
> >
> > When fsck -fy is run on /repl partition then the end result is content of file x is seen in file y.
>
>
>
> --
> : Lars Ellenberg
> : LINBIT | Your Way to High Availability
> : DRBD/HA support and consulting http://www.linbit.com
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> __
> please don't Cc me, but send to list -- I'm subscribed
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20100907/8fb434d0/attachment.htm>