[DRBD-user] drbd & multicast?

[Robert Altnoeder]

> On 06 Apr 2020, at 10:17, Volodymyr Litovka <doka.ua at gmx.com> wrote:
> 
> To avoid this, I'd propose to add additional layer like proxy, which will:
> 
> - reside on every satellite
> - receive data over unicast
> ** thus, drbd code will get minimal changes (now - it sends unicast data
> to 1+ neighbors, after changes - it will send the same unicast to single
> neighbor)
> ** to minimize delay - use local sockets
> - resend it over multicast
> - but manage control traffic (e.g. acknowledgments from remote peers)
> over unicast

This would probably still require many changes in the DRBD kernel module, add another layer of complexity, another component that can fail independently, and makes the system as a whole harder to maintain and troubleshoot.

Delay would probably also be rather unpredictable, because different threads in kernel- and user-space must be activated and paused frequently for the IPC to work, and Linux, as a monolithical kernel, does not offer any specialized mechanisms for direct low-latency context switches/thread activation in a chain of I/O servers like those mechanisms that are found in most microkernels, or at least something in the general direction like e.g. “door calls” in the SunOS kernel (the kernel of the Solaris OS). 

Multicast in DRBD would certainly make sense in various scenarios, but it would probably have to be implemented directly in DRBD.

Anyway, I don’t see that much difference between diskless nodes and nodes with storage. Any one of these nodes always sends write requests to all connected storage nodes, the only difference with diskless nodes is that they also use the replication link for reading data, which storage nodes rather do locally (load-balancing may cause read requests over the network too). So the only thing that would make write performance on a diskless node worse than write performance on a node with local storage would be network saturation due to lots of read requests putting load on the network.

br,
Robert