[DRBD-user] Resource is inconsistent

Lars Ellenberg lars.ellenberg at linbit.com
Sat Apr 16 03:50:43 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Fri, Apr 15, 2011 at 12:55:43PM -0500, Jesse Angell wrote:
> DRBD list,
> 
> I have a two box DRBD installation where the disk on the secondary is stuck as inconsistent status.  The status is:
> version: 8.3.7 (api:88/proto:86-91)
> GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by @prodnfs01secondary.oma.srv.firespring.com, 2010-11-22 15:06:26
>  0: cs:Connected ro:Secondary/Primary ds:Inconsistent/UpToDate A r----
>     ns:0 nr:124479852 dw:124479852 dr:0 al:0 bm:265005 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:163
> 
> If I invalidate the resource on the secondary machine it will sync again and then report as inconsistent.
> 
> I am running the Ubuntu 10.04 LTS drbd package which is version 8.3.7.
> 
> I've attached the following:
> drbd-status.txt - output of cat /proc/drbd on both machines
> syslog-output.txt - the output of syslog after running drbdadm connect r0 on the secondary
> config.txt - the config both machines are using.
> 
> I've been trying to figure this out for days..would appreciate any advice.

You likely want "on-io-error detach;" in drbd.conf.

> Apr 15 12:46:20 prodnfs01primary kernel: [1240665.050426] block drbd0: Got NegAck packet. Peer is in troubles?
> Apr 15 12:46:20 prodnfs01primary kernel: [1240665.050436] block drbd0: Got NegAck packet. Peer is in troubles?
> Apr 15 12:46:20 prodnfs01primary kernel: [1240665.050450] block drbd0: Got NegAck packet. Peer is in troubles?
> Apr 15 12:46:20 prodnfs01primary kernel: [1240665.050455] block drbd0: Got NegAck packet. Peer is in troubles?
> Apr 15 12:46:20 prodnfs01primary kernel: [1240665.050459] block drbd0: Got NegAck packet. Peer is in troubles?
> Apr 15 12:46:35 prodnfs01primary kernel: [1240680.114623] block drbd0: Resync done (total 16 sec; paused 0 sec; 2064 K/sec)
> Apr 15 12:46:35 prodnfs01primary kernel: [1240680.114627] block drbd0:             408 failed blocks


> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.514751] sd 4:0:0:0: [sdc] Unhandled sense code
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.514756] sd 4:0:0:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.514760] sd 4:0:0:0: [sdc] Sense Key : Hardware Error [current]
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.514765] sd 4:0:0:0: [sdc] Add. Sense: Internal target failure
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.514770] sd 4:0:0:0: [sdc] CDB: Write(10): 2a 00 1f 80 5f 58 00 00 a8 00
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.518314] block drbd0: write: error=-5 s=527480280s
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.518317] __ratelimit: 185 callbacks suppressed
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.525954] block drbd0: write: error=-5 s=527480288s
...


> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.685637] sd 4:0:0:0: [sdc] Unhandled sense code
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.685641] sd 4:0:0:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
> Apr 15 12:46:20 prodnfs01secondary kernel: [1239885.685650] sd 4:0:0:0: [sdc] Sense Key : Hardware Error [current]

~100 identical logs of "sdc hardware error" snipped.

It's plain english.

Your sdc is toast.


-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
__
please don't Cc me, but send to list   --   I'm subscribed



More information about the drbd-user mailing list