[DRBD-user] DRBD/GFS2 Stack Trace

Mark Watts m.watts at eris.qinetiq.com
Wed Jan 18 12:07:55 CET 2012

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Not sure if this is caused by DRBD but I've had a CMAN/DRBD/GFS2 cluster stop doing I/O to the DRBD volume overnight with the following stack trace in
syslog:

OS is CentOS 5.7 i386:

kernel-PAE-2.6.18-274.7.1.el5
drbd83-8.3.8-1.el5.centos
kmod-drbd83-PAE-8.3.8-1.el5.centos
gfs2-utils-0.1.62-31.el5

Jan 18 09:04:17 web01 kernel:INFO: task pdflush:294 blocked for more than 120 seconds.
Jan 18 09:04:17 web01 kernel:"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 18 09:04:17 web01 kernel:pdflush       D 0019A7B6  2392   294     27           295   293 (L-TLB)
Jan 18 09:04:17 web01 kernel:f7f68e3c 00000046 506f08e2 0019a7b6 f6f72988 f6f72980 00001070 0000000a
Jan 18 09:04:17 web01 kernel:f7f67aa0 506f9108 0019a7b6 00008826 00000001 f7f67bac c3619544 f7361c80
Jan 18 09:04:17 web01 kernel:f8d863cf 00000000 f8d868c3 00000005 f6d0f8f0 00000000 f669c42c ffffffff
Jan 18 09:04:17 web01 kernel:Call Trace:
Jan 18 09:04:17 web01 kernel:[<f8d863cf>] gdlm_ast+0x0/0x31b [lock_dlm]
Jan 18 09:04:17 web01 kernel:[<f8d868c3>] gdlm_bast+0x0/0xbf [lock_dlm]
Jan 18 09:04:17 web01 kernel:[<f8e0531f>] just_schedule+0x5/0x8 [gfs2]
Jan 18 09:04:17 web01 kernel:[<c062145f>] __wait_on_bit+0x33/0x58
Jan 18 09:04:17 web01 kernel:[<f8e0531a>] just_schedule+0x0/0x8 [gfs2]
Jan 18 09:04:17 web01 kernel:[<f8e0531a>] just_schedule+0x0/0x8 [gfs2]
Jan 18 09:04:17 web01 kernel:[<c06214e6>] out_of_line_wait_on_bit+0x62/0x6a
Jan 18 09:04:17 web01 kernel:[<c043734c>] wake_bit_function+0x0/0x3c
Jan 18 09:04:17 web01 kernel:[<f8e052e2>] gfs2_glock_wait+0x2e/0x66 [gfs2]
Jan 18 09:04:17 web01 kernel:[<f8e1562e>] gfs2_write_inode+0x5c/0x15a [gfs2]
Jan 18 09:04:17 web01 kernel:[<f8e15627>] gfs2_write_inode+0x55/0x15a [gfs2]
Jan 18 09:04:17 web01 kernel:[<c049576e>] __writeback_single_inode+0x199/0x2a5
Jan 18 09:04:17 web01 kernel:[<f8dd8b08>] drbd_congested+0x1a3/0x235 [drbd]
Jan 18 09:04:17 web01 kernel:[<c0495b5e>] sync_sb_inodes+0x17e/0x221
Jan 18 09:04:17 web01 kernel:[<c0495dad>] writeback_inodes+0x6a/0xb0
Jan 18 09:04:17 web01 kernel:[<c045d7d6>] wb_kupdate+0xcb/0x130
Jan 18 09:04:17 web01 kernel:[<c045dbf7>] pdflush+0x0/0x1a1
Jan 18 09:04:17 web01 kernel:[<c045dd02>] pdflush+0x10b/0x1a1
Jan 18 09:04:17 web01 kernel:[<c045d70b>] wb_kupdate+0x0/0x130
Jan 18 09:04:17 web01 kernel:[<c043725a>] kthread+0xc0/0xee
Jan 18 09:04:17 web01 kernel:[<c043719a>] kthread+0x0/0xee
Jan 18 09:04:17 web01 kernel:[<c0405c87>] kernel_thread_helper+0x7/0x10
Jan 18 09:04:17 web01 kernel:=======================

Any hints?

- -- 
Mark Watts BSc RHCE
Senior Systems Engineer, MSS Secure Managed Hosting
www.QinetiQ.com
QinetiQ - Delivering customer-focused solutions
GPG Key: http://www.linux-corner.info/mwatts.gpg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (GNU/Linux)
Comment: Using GnuPG with CentOS - http://enigmail.mozdev.org/

iEYEARECAAYFAk8WqAsACgkQBn4EFUVUIO3pwACgrBZH5Aj8WgP1S7ipOaFz2xSK
tsIAoOK/ENCao2+GdXtsv0wsbkISHrUu
=mbSb
-----END PGP SIGNATURE-----



More information about the drbd-user mailing list