[DRBD-user] Spin_Lock timeout in DRBD during heavy load

[Lars Ellenberg]

On Thu, May 16, 2019 at 04:49:08PM +0200, stefan.flothkoetter wrote:
> Hi!
> 
> I'm frequently seeing spin_lock timeouts during heavy load caused by a sync:
> Using current CentOS7 with Kernel 3.10.0-957.12.1.el7.x86_64 and DRBD
> 9.0.16-1:
> 
> [ 2641.853381]  [<ffffffff9156b6e7>] _raw_spin_lock_irqsave+0x37/0x40
> [ 2641.927774]  [<ffffffffc0b7357f>] drbd_rs_complete_io+0x3f/0x160 [drbd]
> [ 2642.007354]  [<ffffffffc0b4d8ac>] drbd_endio_write_sec_final+0x3cc/0x460 [drbd]
> [ 2642.095228]  [<ffffffffc0b4db18>] drbd_peer_request_endio+0x1d8/0x420 [drbd]
> [ 2642.179992]  [<ffffffff9107fc87>] bio_endio+0x67/0xb0
> [ 2642.240899]  [<ffffffffc0324125>] dec_pending+0x1b5/0x350 [dm_mod]
> [ 2642.315269]  [<ffffffff90fb96c7>] ? mempool_free_slab+0x17/0x20
> [ 2642.386547]  [<ffffffffc0324df1>] clone_endio+0x71/0x100 [dm_mod]
> [ 2642.459884]  [<ffffffff9107fc87>] bio_endio+0x67/0xb0
> [ 2642.520783]  [<ffffffff91145a40>] blk_update_request+0x90/0x360
> [ 2642.592064]  [<ffffffff912dacd4>] scsi_end_request+0x34/0x1e0
> [ 2642.661259]  [<ffffffff912db048>] scsi_io_completion+0x168/0x6a0
> [ 2642.733575]  [<ffffffff912d9a41>] ? scsi_done+0x21/0x60
> [ 2642.796545]  [<ffffffff912d04fc>] scsi_finish_command+0xdc/0x140
> [ 2642.868855]  [<ffffffff912da592>] scsi_softirq_done+0x132/0x160
> [ 2642.940127]  [<ffffffff9114d366>] blk_done_softirq+0x96/0xc0
> [ 2643.008288]  [<ffffffff90ea0f45>] __do_softirq+0xf5/0x280
> [ 2643.073335]  [<ffffffff915792ec>] call_softirq+0x1c/0x30
> [ 2643.137344]  [<ffffffff90e2e675>] do_softirq+0x65/0xa0
> [ 2643.199280]  [<ffffffff90ea12c5>] irq_exit+0x105/0x110
> [ 2643.261216]  [<ffffffff9157a5a6>] do_IRQ+0x56/0xf0
> [ 2643.318999]  [<ffffffff9156c362>] common_interrupt+0x162/0x162
> [ 2643.389221]  <EOI>  [<ffffffffc0b6251b>] ? receive_Data+0x78b/0x18f0 [drbd]
> [ 2643.473042]  [<ffffffffc0b6250e>] ? receive_Data+0x77e/0x18f0 [drbd]

Supposedly fixed with 9.0.18, more specifically with
7ce7cac6 drbd: fix potential spinlock deadlock on device->al_lock

-- 
: Lars Ellenberg
: LINBIT | Keeping the Digital World Running
: DRBD -- Heartbeat -- Corosync -- Pacemaker

DRBD® and LINBIT® are registered trademarks of LINBIT
__
please don't Cc me, but send to list -- I'm subscribed