[DRBD-user] local WRITE IO error sector 21776+1016 on dm-2

[Roland JARRY]

Hello,

I answer to myself (and others if it can help !).

I seems to be the same issue described by Lars
(http://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html)
: WRITE SAME not supported by my hardware (HPE Smart Array P816i-a SR
Gen10 on HP ProLiant DL380 Gen10).

I saw a lot of posts about this issue but didn't see messages about
WRITE SAME on my log (neither by DRBD, nor by device manager LVM).

Work around proposed by Lars works.

Maybe this one can also be used :
https://chris.hofstaedtler.name/blog/2016/10/kernel319plus-3par-incompat.html
finding before ATTRS{rev} property of disks.

Roland.


On 23/07/2019 10:53, Roland JARRY wrote:
> Hello,
>
> I have an issue mounting drbd 8.4.11-1 resources on a kernel
> 4.9.0-9-amd64 (debian 9.9). I have this error message : block drbd3:
> local WRITE IO error sector 21776+1016 on dm-2
>
> Then, the resource becomes diskless.
>
> Here are the settings of the resource :
>
> root at srv-pg-sav-p:~# cat /etc/drbd.d/vgbackup-lv-back3.res
> resource vgbackup-lv-back3 {
>   net {
>     allow-two-primaries;
>   }
>   startup {
>     wfc-timeout 120;
>     degr-wfc-timeout 120;
>   }
>
>   volume 0 {
>     device    /dev/drbd3;
>     #meta-disk internal;
>     meta-disk /dev/vgbackup/lv-md-back3;
>     disk      /dev/vgbackup/lv-back3;
> }
>
>   on srv-pg-sav-p {
>     address   192.168.8.221:7803;
>
>   }
>   on srv-pg-sav-s {
>     address   192.168.8.222:7803;
>   }
> }
>
> I've changed meta-disk internal to external lv device to have more space
> (1GB), but I have the same issue :
>
> root at srv-pg-sav-p:~# lvs
>   LV          VG       Attr       LSize  Pool Origin Data%  Meta%  Move
> Log Cpy%Sync Convert
>   lv-back1    vgbackup -wi-ao----
> 21.00t                                                   
>   lv-back2    vgbackup -wi-ao----
> 21.00t                                                   
>   lv-back3    vgbackup -wi-a-----
> 21.00t                                                   
>   lv-md-back3 vgbackup -wi-a-----  1.00g   
>
> I have 3 resources of same size. 2 works right now and not the 3rd. And
> I had the same issue before with 2 first resources.
>
> I notice that the error is on the same sector on each resource and at
> each time. Is there a limitation somewhere ?
>
> Here is more log :
>
> Jul 23 10:23:52 srv-pg-sav-p kernel: [1532462.342531] EXT4-fs (drbd3):
> mounted filesystem with ordered data mode. Opts: (null)
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860138] block drbd3: local
> WRITE IO error sector 21776+1016 on dm-2
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860146] block drbd3: disk(
> UpToDate -> Failed )
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860177] block drbd3: Local
> IO failed in __req_mod. Detaching...
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868306] block drbd3:
> helper command: /sbin/drbdadm pri-on-incon-degr minor-3
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868356] block drbd3: IO
> ERROR: neither local nor remote data, sector 21776+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.876611] block drbd3: IO
> ERROR: neither local nor remote data, sector 21784+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.881934] block drbd3:
> helper command: /sbin/drbdadm pri-on-incon-degr minor-3 exit code 0 (0x0)
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.885020] block drbd3: IO
> ERROR: neither local nor remote data, sector 21792+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894204] block drbd3: 21 TB
> (5637144528 bits) marked out-of-sync by on disk bit-map.
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894207] block drbd3: disk(
> Failed -> Diskless )
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894220] block drbd3: IO
> ERROR: neither local nor remote data, sector 21800+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.902118] block drbd3: IO
> ERROR: neither local nor remote data, sector 21808+8
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637796] block drbd3: 122
> messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637802] block drbd3: IO
> ERROR: neither local nor remote data, sector 22548840448+8
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.648265] Buffer I/O error
> on dev drbd3, logical block 2818605056, lost sync page write
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.658056] JBD2: Error -5
> detected when updating journal superblock for drbd3-8.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.668068] Aborting journal
> on device drbd3-8.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.678189] Buffer I/O error
> on dev drbd3, logical block 2818605056, lost sync page write
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.688556] JBD2: Error -5
> detected when updating journal superblock for drbd3-8.
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213355] block drbd3: 1
> messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213361] block drbd3: IO
> ERROR: neither local nor remote data, sector 45097156480+8
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.222815] block drbd3: IO
> ERROR: neither local nor remote data, sector 45097156592+8
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.232250] block drbd3: IO
> ERROR: neither local nor remote data, sector 0+8
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.241269] block drbd3: IO
> ERROR: neither local nor remote data, sector 8+8
>
> Kind regards.
>