[DRBD-user] VMs randomly become to read-only mode

[kvaps]

I was wrong about few facts:

- if VM has many disks all of them will become readonly
I just found the VM with the same symptoms but only one disk were in
read-only mode

- virsh domfsthaw returns error
for this vm it is working correctly, but no change and disk is still
not writable afterwards

I can also append that usually this is happening after the snapshot
operations on drive (we are using linstor with lvmthin backend)

drbd version:

version: 9.0.23-1 (api:2/proto:86-116)
GIT-hash: d16bfab7a4033024fed2d99d3b179aa6bb6eb300 build by
@runner-cpuncbtu-project-69-concurrent-15tjkf, 2020-06-09 23:46:40
Transports (api:16): tcp (9.0.23-1)

Best Regards,
Andrei Kvapil

On Tue, Jul 21, 2020 at 5:12 PM kvaps <kvapss at gmail.com> wrote:
>
> Hi, time to time we're facing with the weird situations when VMs are
> coming to readonly mode:
>
> Eg. /dev/vda is allowed to read but not for write:
>
> (initramfs) dd if=/dev/vda of=/tmp/1 bs=1 count=1
> 1+0 records in
> 1+0 records out
> (initramfs) dd if=/tmp/1 if=/dev/vda bs=1 count=1
> [ 1335.967438] print_req_error: I/O error, dev vda, sector 0
> [ 1335.969843] Buffer I/O error on dev vda, logical block 0, lost
> async page write
> 1+0 records in
> 1+0 records out
> (initramfs)
>
> Today about 8 VMs in different clusters become to this state, so I had
> a time to experiment with them:
>
> - dmesg messages are clean
> - drbd state is seems ok:
>
> one-vm-604-disk-2 role:Primary
>   disk:UpToDate
>   m1c10 role:Secondary
>     peer-disk:UpToDate
>   m1c7 role:Secondary
>     peer-disk:Diskless
>
> - from the host side the disk is accessible and writable
> - if VM has many disks all of them will become readonly
> - simple reboot does not solves the problem, poweroff then resume does.
> - virsh domfsthaw returns:
>
> error: Unable to thaw filesystems
> error: Guest agent is not responding: QEMU guest agent is not connected
>
> Has anyone the similar cases?
>
> Best Regards,
> Andrei Kvapil