[DRBD-user] DRBD serious locking due to TOE

Florian Haas florian.haas at linbit.com
Fri Dec 14 14:43:53 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

ethtool -K tx off eth<num>
ethtool -K rx off eth<num>

This is based on the assumption that your issue is not one with TOE (TCP 
Offload Engine, a term that I've only seen applied to iSCSI HBAs and 10GbE 
cards thus far), but with TCP checksum offloading (a feature present on 
virtually all contemporary Ethernet NICs).

And, this is supported by the bnx2 driver.


On Friday 14 December 2007 11:17:34 Ben Clewett wrote:
> Dear DRBD,
> I have a repeatable problem with DRBD 8.2.1 where it locks up, and the
> replication ability falls by several orders of magnitude.  This is the
> same as the problem reported by Ben Lavender on 2007-08-29.
> Ben identified the problem as due to the TOE protocol on his DELL
> network card.  Our HP network cards (NetXtreme II BCM5708 1000Base-SX)
> use the same Broadcom chipset, but unlike the DELL card, the HP card
> provides no mechanism to disable TOE.  Or at least no published
> mechanism in the BIOS or available to Linux, and no jumpers on the PCB.
> The problem occurs under heavy loading.  The NIC's ability to handle TCP
> packets falls to about a tenth of it's normal rate, which is normally
> 100MB/sec on our set-up.  Therefore rendering DRBD and our MySql
> database unusable for a few minutes.
> I would like to ask if there is anything that can be done in DRBD to get
> round this problem, like for instance using UDP instead of TCP, or some
> bug-fix for TOE which any member may know about?
> If this is not the case we will have to replace our NIC's, which is
> really not something we want to do, since all HP NIC's for HP servers
> seem to have the same chipset.
> Any advise would be extremely welcome!
> Regards,
> Ben Clewett.

: Florian G. Haas
: LINBIT Information Technologies GmbH
: Vivenotgasse 48, A-1120 Vienna, Austria

More information about the drbd-user mailing list