Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Tue, Mar 08, 2011 at 02:55:39PM -0800, Sean McCreadie wrote:
> Hello,
>
> I have a basic question regarding the way drbd replication works. I
> am using drbd 8.3.8 to mirror a volume that I am exporting using IET
> iscsi-target. I am using protocol C and I have the iscsi-target
> exporting the volume using write-through IOMode.
>
> Can I safely enable write-back mode on the iscsi-target with DRBD?
> (independent of the inherent risks associated with write-back anyways,
> I have BBWC controller and UPS on server)
>
> Will the memory buffering of writes affect data integrity on the peer
> node? Should or can I use another DRBD protocol?
>
> I obviously am new to DRBD. Thanks in advance for the help.
DRBD will only replicate writes it knows about.
DRBD does not know about writes that only reached the linux page cache
(that is, the "write back cache" of the linux iscsi target).
So if you happen to have 90 GB of RAM all used as write back cache
for your target, and they are all dirty, and now you get a node crash,
those 90 GB worth of modifications are LOST.
No block device ever knew about them,
DRBD was not told to replicate them,
DRBD was not told to write them to stable storage.
Those modifications have been in volatile cache only, and are just lost.
And whether or not you use DRBD does not affect things here. The amount
of data lost is the same as with a non-DRBD setup: every changes that
lived only in the page cache, and have not yet been written to stable
storage, are lost on node crash/power failure.
That is the nature of "write back" caching:
*pretend* something had already been written, that actually will only be
written to stable storage some time later.
Try to imagine the stack:
[initiator]
|
[target]
|
[page-cache] (always volatile)
|
. . . . . . . . . . . . . . . . .
|
|
[DRBD] ================== [DRBD]
| |
[disk-cache] [disk-cache]
[stable-storage] [stable-storage]
Make sure that [disk-cache] is non-volatile.
That is what the BB in BBWC stands for.
Or disable that cache.
Changes that did not make it below the dotted line
are lost on node crash/power fail.
Changes that did make it below the dotted line,
but have not been confirmed to upper layers as written yet,
_may_ be lost on node crash/power fail.
Depending on the DRBD protocol, even changes that have been confirmed as
written (made it down AND back) may be lost after failover, in case they
- have not reached the remote disk yet
(protocol A or B, and simultaneous (power) failure of both nodes)
- have not reached the remote node yet (protocol A only)
Hope that helps to understand what will,
and what may, be lost in which scenario.
Using write-back is independend of DRBD,
and
will lose data
on node crash/power failure.
Modifications that reach DRBD will be on both nodes,
within the limitations of the replication protocol in use.
--
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com