[DRBD-user] TRIM/discard leads to secondary becoming diskless

Olav Morken olavmrk at gmail.com
Mon Oct 5 20:27:29 CEST 2015

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On Mon, Oct 05, 2015 at 16:49:11 +0200, Lars Ellenberg wrote:
> On Sun, Oct 04, 2015 at 08:36:37PM +0200, Olav Morken wrote:
> > Hi,
> > 
> > I'm testing DRBD on a couple of virtual machines running Debian
> > Jessie, and noticed that every time a discard request is sent to the
> > DRBD-device, the secondary DRBD host becomes diskless:
> > 
> > > [  121.779367] block drbd2: discard: error=-95 s=0s
> 
> Please also reproduce with the out of tree DRBD 8.4.
> 
> Also, what exactly is the IO stack below DRBD?
> If the lower level device below the receiving side does not support
> discards, DRBD is supposed to fall back to zero-out.
> 
> If the lower level device below the sending (Primary) side does not
> support discards, DRBD supposedly does not even announce discard to
> upper layers, and these requests should never reach us.
> 
> If the lower level devices pretend to support discards,
> but then fail with EOPNOTSUP, on the Primary side, we can propagate this
> error to upper layers without detaching, assuming the upper layer will
> then "deal with it appropriately".

Hi,

thanks for the explanation about how this is supposed to work! I had
missed the part in the code where it was only supposed to enable this
if the underlying block device supports discard.

Once I started looking at my storage stack, I discovered that I had
created the LVM-volumes in the virtual machines incorrectly. I had
attempted to create a thinly provisioned LVM volume, but what I
instead had created was a LVM storage pool for thinly provisioned
volumes.

That volume type appears to do something strange wrt. discard
support. Since that volume type isn't supposed to be used for anything
other than creating LVM volumes, I can't really fault LVM for that.

In any case, once I recreated the LVM-volumes as normal LVM volumes,
discard was correctly detected as not supported:

> # blkdiscard /dev/drbd0
> blkdiscard: /dev/drbd0: BLKDISCARD ioctl failed: Operation not supported

Afterwards, I recreated the volumes as proper thin volumes, and now
the discard operation succeeds:

> # lvs vg/dvb-ch-01
>   LV        VG   Attr       LSize Pool     Origin Data%  Meta%  Move  Log Cpy%Sync Convert
>   dvb-ch-01 vg   Vwi-aotz-- 4.00g thinpool        1.61
> # blkdiscard /dev/drbd/by-res/ch-01
> # lvs vg/dvb-ch-01
>   LV        VG   Attr       LSize Pool     Origin Data%  Meta%  Move Log Cpy%Sync Convert
>   dvb-ch-01 vg   Vwi-aotz-- 4.00g thinpool        0.00

(Though discard support wasn't really my goal of using thin volumes in
the first place, it is a nice bonus.)

Best regards,
Olav Morken



More information about the drbd-user mailing list