[DRBD-user] VMs randomly become to read-only mode

[kvaps]

Hi, time to time we're facing with the weird situations when VMs are
coming to readonly mode:

Eg. /dev/vda is allowed to read but not for write:

(initramfs) dd if=/dev/vda of=/tmp/1 bs=1 count=1
1+0 records in
1+0 records out
(initramfs) dd if=/tmp/1 if=/dev/vda bs=1 count=1
[ 1335.967438] print_req_error: I/O error, dev vda, sector 0
[ 1335.969843] Buffer I/O error on dev vda, logical block 0, lost
async page write
1+0 records in
1+0 records out
(initramfs)

Today about 8 VMs in different clusters become to this state, so I had
a time to experiment with them:

- dmesg messages are clean
- drbd state is seems ok:

one-vm-604-disk-2 role:Primary
  disk:UpToDate
  m1c10 role:Secondary
    peer-disk:UpToDate
  m1c7 role:Secondary
    peer-disk:Diskless

- from the host side the disk is accessible and writable
- if VM has many disks all of them will become readonly
- simple reboot does not solves the problem, poweroff then resume does.
- virsh domfsthaw returns:

error: Unable to thaw filesystems
error: Guest agent is not responding: QEMU guest agent is not connected

Has anyone the similar cases?

Best Regards,
Andrei Kvapil