[DRBD-user] dettach on io-error failed on pending i/o ops

Lars Ellenberg Lars.Ellenberg at linbit.com
Fri Dec 23 13:57:50 CET 2005

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

/ 2005-12-23 11:47:23 +0100
\ David Engraf:
> > Von: drbd-user-bounces at lists.linbit.com [mailto:drbd-user-
> > bounces at lists.linbit.com] Im Auftrag von Lars Ellenberg
> > Gesendet: Freitag, 23. Dezember 2005 11:02
> > An: drbd-user at lists.linbit.com
> > Betreff: Re: [DRBD-user] dettach on io-error failed on pending i/o ops
> > 
> > / 2005-12-23 10:07:26 +0100
> > \ David Engraf:
> > > I have a fileserver with drbd in primary mode and a second computer in
> > > secondary mode. Now I plug off my hard disk from the primary and want
> > > drbd to detach the hd and go to diskless mode.
> > > Everything works fine if there are no pending i/o operations, but when
> > > there are pending i/o ops drbd waits until ALL operations are
> > > finished. The problem is that these operations are running in
> > > timeout, so the drbd waits about a few minutes until all are
> > > finished.  While drbd is waiting I can read from the fileserver
> > > but not write.  I think drbd_io_error in drbd_main.c should cancel
> > > all pendig i/o requests and give the device immediately free.
> > 
> > we cannot possibly cancel requests that we submitted to the io stack
> > below us. what if we cancel them, and then later the driver below us
> > cancels them, too, but they no longer exist??
> > 
> > fix the lower level driver to fail these requests faster.
> >
> When the application cancels i/o requests, the drbd can also cancel
> the pending requests over the net and should, if there are any more
> requests on the local device, cancel them too.

I don't see it is that easy.
If you see something which I happen to overlook, please provide a patch,
or "prove of concept" or the like.

> The problem is as long as there are any pending requests, I can't
> write to the fileserver...is this ok??

this is not coded in drbd.
I did not observe this either, when I did "detach tests".

> I think drbd can operate in diskless state, so why don't use this
> feature?

we use it.

some of the relevant code is in drbd_io_error() in drbd_main.c, and as I
read the code, there should be no more than one second between the two
messages WARN("Notified peer that my disk is broken.\n");
and either WARN("Not releasing backing storage device.\n");
or WARN("Releasing backing storage device.\n");
unless this is time spent in drbd_md_write(mdev)...
and I won't throw away this call. it should not block anything, either.
but maybe we can at least add some "fail fast do not retry" flag here,
if the lower level drivers support it.

it likely is not drbd who blocks further IO, but the file system (which
waits for some journal write to complete).

imho, the solution really is to make the lower level driver
fail requests faster.

I may be wrong of course, maybe there is a simple solution on the
drbd layer, even if it is not our fault. but currently I don't see it...

: Lars Ellenberg                                  Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH            Fax +43-1-8178292-82 :
: Schoenbrunner Str. 244, A-1120 Vienna/Europe   http://www.linbit.com :
please use the "List-Reply" function of your email client.

More information about the drbd-user mailing list