Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Fri, 2008-12-13 at 17:11 +0100, Lars Ellenberg wrote: > appart from running many processes doing direct io to the same > block, there is not much I can think of that may produce these > concurrent writes. I am doing direct IO (via kvm's cache=off option). There is only the one process, but I believe it simulates AIO using glibc's thread-based AIO implementation. Since the guest (debian etch) is using SCSI TCQ, it could in theory write the same block many times. No idea why it would do that though. I've also no idea why running a verify would trigger it, if that is something more than mere coincidence. > maybe some disk image accidentally mounted on the host, > while simultaneously being used from within a virtual machine guest? The drbd device in question is only used by the 1 KVM process. > or some "optimization" taking place on the virtual disk driver? KVM's block code (inherited from qemu) in block-raw-posix.c looks pretty straightforward. No optimizations more fancy than AIO... > can you do the ps -eo | grep magic on the other node as well please? # ps -eo pid,state,wchan:30,cmd | grep -e drbd -e D PID S WCHAN CMD 3002 S select /usr/local/bin/qemu-system-x86_64 -m 512 -drive file=/dev/drbd23,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name root_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=virtio -net tap -monitor unix:/tmp/VM_root_vm_0,server,nowait -tdf -daemonize -vnc :0,password 4696 S sk_wait_data [drbd1_receiver] 4698 S sk_wait_data [drbd1_asender] 5190 S sk_wait_data [drbd2_receiver] 5193 S sk_wait_data [drbd3_receiver] 5196 S sk_wait_data [drbd4_receiver] 5199 S sk_wait_data [drbd5_receiver] 5202 S sk_wait_data [drbd6_receiver] 5205 S sk_wait_data [drbd7_receiver] 5208 S sk_wait_data [drbd8_receiver] 5211 S sk_wait_data [drbd9_receiver] 5214 S sk_wait_data [drbd10_receiver] 5217 S sk_wait_data [drbd11_receiver] 5220 S sk_wait_data [drbd12_receiver] 5223 S sk_wait_data [drbd13_receiver] 5226 S sk_wait_data [drbd14_receiver] 5229 S sk_wait_data [drbd15_receiver] 5232 S sk_wait_data [drbd16_receiver] 5235 S sk_wait_data [drbd17_receiver] 5238 S sk_wait_data [drbd18_receiver] 5241 S sk_wait_data [drbd19_receiver] 5244 S sk_wait_data [drbd20_receiver] 5249 S sk_wait_data [drbd21_receiver] 5255 S sk_wait_data [drbd22_receiver] 5258 S sk_wait_data [drbd23_receiver] 5261 S drbd_wait_peer_seq [drbd24_receiver] 5262 S sk_wait_data [drbd4_asender] 5263 S sk_wait_data [drbd5_asender] 5264 S sk_wait_data [drbd3_asender] 5265 S sk_wait_data [drbd6_asender] 5266 S sk_wait_data [drbd7_asender] 5267 S sk_wait_data [drbd9_asender] 5269 S sk_wait_data [drbd8_asender] 5270 S sk_wait_data [drbd10_asender] 5271 S sk_wait_data [drbd11_asender] 5272 S sk_wait_data [drbd12_asender] 5273 S sk_wait_data [drbd13_asender] 5274 S sk_wait_data [drbd14_asender] 5275 S sk_wait_data [drbd15_asender] 5276 S sk_wait_data [drbd17_asender] 5277 S sk_wait_data [drbd16_asender] 5278 S sk_wait_data [drbd18_asender] 5279 S sk_wait_data [drbd21_asender] 5280 S sk_wait_data [drbd19_asender] 5281 S sk_wait_data [drbd20_asender] 5282 S sk_wait_data [drbd22_asender] 5283 S sk_wait_data [drbd23_asender] 5284 S sk_wait_data [drbd24_asender] 5285 S sk_wait_data [drbd2_asender] 5408 S down_interruptible [drbd1_worker] 5417 S down_interruptible [drbd2_worker] 5426 S down_interruptible [drbd3_worker] 5435 S down_interruptible [drbd4_worker] 5444 S down_interruptible [drbd5_worker] 5453 S down_interruptible [drbd6_worker] 5462 S down_interruptible [drbd7_worker] 5471 S down_interruptible [drbd8_worker] 5480 S down_interruptible [drbd9_worker] 5489 S down_interruptible [drbd10_worker] 5498 S down_interruptible [drbd11_worker] 5507 S down_interruptible [drbd12_worker] 5516 S down_interruptible [drbd13_worker] 5525 S down_interruptible [drbd14_worker] 5534 S down_interruptible [drbd15_worker] 5543 S down_interruptible [drbd16_worker] 5552 S down_interruptible [drbd17_worker] 5561 S down_interruptible [drbd18_worker] 5570 S down_interruptible [drbd19_worker] 5579 S down_interruptible [drbd20_worker] 5588 S down_interruptible [drbd21_worker] 5597 S down_interruptible [drbd22_worker] 5606 S down_interruptible [drbd23_worker] 5615 S down_interruptible [drbd24_worker] 6101 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd3,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :3,password 6134 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd5,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :5,password 6268 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd8,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :8,password 6498 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd9,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :9,password 6518 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd10,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :10,password 6541 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd11,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :11,password 6890 S select qemu-system-x86_64 -m 512 -drive file=/dev/drbd4,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=e1000 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :4,password 8968 S select /usr/local/bin/qemu-system-x86_64 -m 512 -drive file=/dev/drbd2,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=rtl8139 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :2,password 16302 R select /usr/local/bin/qemu-system-x86_64 -m 512 -drive file=/dev/drbd1,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=rtl8139 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :1,password 29430 S select /usr/local/bin/qemu-system-x86_64 -m 512 -drive file=/dev/drbd7,if=scsi,cache=off,boot=on -drive if=ide,index=2,media=cdrom -usbdevice tablet -name xxxx_vm_0 -net nic,macaddr=xx:xx:xx:xx:xx:xx,model=rtl8139 -net tap -monitor unix:/tmp/VM_xxxx_vm_0,server,nowait -tdf -daemonize -vnc :6,password