Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
hi:
I finally got the resource synced again. it seems some kind of
drbd race condition.
I have another 1TB resource. when I verify/resync the 32GB resource,
the verify process of the 1TB resource is still running. the 32GB can
not get verify/resync correctly.
the 32GB resource is a volume, there is another 10GB resource
volume belong to the same resource, if the two volume verify together,
the 32GB can not get verify/resync corretcly.
when there is no other resource/volume is under verify, the 32GB
volume finally get verify/resync correctly.
so it seems under 8.4.7, I should only verify/resync one resource
volume at the same time, or bad things may happen.
I have 25 resource volumes, and after I run "drbdadm verify all", I
got about 10 volumes with oos, that's terrible...
2016-05-19 15:58 GMT+08:00 Veit Wahlich <cru.lists at zodia.de>:
> Hi,
>
> well, if it still occurs while the resources are not accessed and thus
> no data is transferred at all except for the resync and verify, I
> suspect a surface or mapping related storage hardware/firmware issue to
> be the culprit, as this would also explain that this issue occurs on the
> same resources again and again while never on others.
>
> In other thoughts I'd suspect that a rogue process on the host system
> alters the backing devices directly, e.g. LVM accessing some DRBD
> resources' backing devices. You might want to use the "pvs" command to
> verify that LVM does not incorporate the backing devices at all.
>
> Regards,
> // Veit
>
> Am Donnerstag, den 19.05.2016, 10:09 +0800 schrieb d tbsky:
>> Hi:
>>
>> it is not ssd. it is just two 2TB sata hard disks. mdadm is
>> checked every week and I don't see any error report under dmesg.
>> there are 20 VMs running above it and they seems normal. but I wonder
>> it will be normal again after so many verify/resync. so I just pick a
>> test-vm to try. still trying the config options to see if I can get it
>> resync. I can also use dd to resync it but then the problem may
>> disappear and I won't know what happened.
>>
>> any suggestions to find out what happened?
>>
>> 2016-05-19 4:50 GMT+08:00 Veit Wahlich <cru.lists at zodia.de>:
>> > Are you utilising SSDs?
>> >
>> > Is the kernel log (dmesg) clean from errors on the backing devices (also mdraid members/backing devices)?
>> >
>> > Did you verify the mdraid array consistency and are the array's members in sync?
>> >
>> >
>> > -------- Ursprüngliche Nachricht --------
>> > Von: d tbsky <tbskyd at gmail.com>
>> > Gesendet: 18. Mai 2016 18:27:00 MESZ
>> > An: Veit Wahlich <cru.lists at zodia.de>
>> > CC: drbd-user at lists.linbit.com
>> > Betreff: Re: [DRBD-user] drbdadm verify always report oos
>> >
>> > Hi:
>> > I shutdown the vm when I found the strange behavior. so the drbd
>> > is resync under idle situation. I try to play with config options and
>> > resync about 15 times, still can not get verify report 0 oos.
>> >
>> > I have about 10 resource which has verify problem. but it's strange
>> > that some resources are ok. the largest resource is about 1T and it is
>> > ok. the resource I am testing now is only 32G.
>> >
>> > the host structure is: two sata disk (mdadm raid 1) -> lvm -> drbd
>> >
>> > the host has ecc ram so the memory is ok. the most confusing part
>> > is the resync data is very small, it is only a few kilo bytes and sync
>> > done in 1 seconds. I don't know why it can not be synced correctly. I
>> > also try to run command "md5sum /dev/drbdX" at both node to check.
>> > they are indeed different.
>> >
>> >
>> > _______________________________________________
>> > drbd-user mailing list
>> > drbd-user at lists.linbit.com
>> > http://lists.linbit.com/mailman/listinfo/drbd-user
>> _______________________________________________
>> drbd-user mailing list
>> drbd-user at lists.linbit.com
>> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user