[DRBD-user] "local disk flush failed with status -5" on LVM

[Stephane Jourdan]

Hi Philipp,

So here is the DRBD setup :

~# drbdsetup /dev/drbd1 show
disk {
        size                    0s _is_default; # bytes
        on-io-error             pass_on _is_default;
        fencing                 dont-care _is_default;
}
net {
        timeout                 60 _is_default; # 1/10 seconds
        max-epoch-size          2048 _is_default;
        max-buffers             2048 _is_default;
        unplug-watermark        128 _is_default;
        connect-int             10 _is_default; # seconds
        ping-int                10 _is_default; # seconds
        sndbuf-size             131070 _is_default; # bytes
        ko-count                0 _is_default;
        allow-two-primaries;
        after-sb-0pri           discard-zero-changes;
        after-sb-1pri           discard-secondary;
        after-sb-2pri           disconnect _is_default;
        rr-conflict             disconnect _is_default;
        ping-timeout            5 _is_default; # 1/10 seconds
}
syncer {
        rate                    700000k; # bytes/second
        after                   -1 _is_default;
        al-extents              127 _is_default;
}
protocol C;
_this_host {
        device                  "/dev/drbd1";
        disk                    "/dev/VG-FULL/dbslash";
        meta-disk               internal;
        address                 10.2.10.2:7790;
}
_remote_host {
        address                 10.2.10.1:7790;
}

and here are some generic /var/log/syslog outputs:
May 14 17:51:38 vlab02 kernel: [110343.557916] drbd1: local disk flush 
failed with status -5
May 14 17:51:38 vlab02 kernel: [110343.561781] drbd1: local disk flush 
failed with status -5
May 14 17:51:38 vlab02 kernel: [110343.562510] drbd1: local disk flush 
failed with status -5
May 14 17:51:38 vlab02 kernel: [110343.565992] drbd1: local disk flush 
failed with status -5
May 14 17:51:38 vlab02 kernel: [110343.566476] drbd1: local disk flush 
failed with status -5

Some background information : there is 2 machines, whose hard-disks are 
mounted on LVM and as you can see in the DRBD config, and the base 
device for DRBD is a logical volume. On this DRBD disk is installed a 
Xen virtual machine, it's its rootFS.
Everything is really working great, with great Gigabit speed and sync 
functions.

This is the exact same setup on both machines, same hardware, same packages.

The errors shown above arrive simply by ssh'ing into the virtual 
machine, or creating files, etc.

By the way, I'm using Ubuntu 8.04 with the Xen kernel, and a  v8.0.12 DRBD.

Thank you a lot for your help !

Cheers,
Stef


Philipp Reisner wrote:
> Am Montag, 12. Mai 2008 17:33:19 schrieb Stephane Jourdan:
>   
>> Hi,
>>
>> I'm currently using DRBD over LVM and getting the same error message
>> than http://lists.linbit.com/pipermail/drbd-user/2008-May/009287.html
>>
>> Upgrading the kernel to 2.6.25 is not an option for now (running
>> Ubuntu/2.6.24)
>> But even using the latest sources from git or v8.0.12 and using the 2
>> new options to disable flushes, I get the "disk flush" failures .
>>
>> As per the last message posted on this topic, is this kernel bug fixed
>> by the workaround you provided ? Or is it, but not in the case of a
>> DRBD-over-LVM setup (and then, only upgrading the kernel would help) ?
>>
>> Thanks for your help!
>>
>>     
>
> Hi Stef,
>
> Can you please post the output of drbdsetup /dev/drbdX show,
> and and excerpt of the kernel logs with the disk flush failures ?
>
> Thanks!
> -phil
>   

-- 
Stephane Jourdan
ReservIT
sjourdan at reservit.com