[DRBD-user] iozone test block on drbd

Mia Lueng xiaozunvlg at gmail.com
Thu May 23 07:23:20 CEST 2013

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


there are some error messages in dmesg:

INFO: task flush-253:5:32096 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-253:5   D 0000000000000002     0 32096      2 0x00000080
 ffff8803b1e339b0 0000000000000046 0000000000000000 ffff88002c292920
 ffff8803b1e33960 ffffffff81127e77 0000000300000001 0000000000000002
 ffff8803a0911038 ffff8803b1e33fd8 000000000000f4e8 ffff8803a0911038
Call Trace:
 [<ffffffff81127e77>] ? ____pagevec_lru_add+0x167/0x180
 [<ffffffffa0168f6a>] start_this_handle+0x25a/0x4d0 [jbd2]
 [<ffffffff8115e5db>] ? cache_alloc_refill+0x15b/0x240
 [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa01693c5>] jbd2_journal_start+0xb5/0x100 [jbd2]
 [<ffffffffa018b035>] ? ext4_meta_trans_blocks+0x75/0xf0 [ext4]
 [<ffffffffa01a5218>] ext4_journal_start_sb+0x58/0x90 [ext4]
 [<ffffffffa018eebc>] ext4_da_writepages+0x27c/0x660 [ext4]
 [<ffffffff81054754>] ? find_busiest_group+0x244/0xb20
 [<ffffffff81126351>] do_writepages+0x21/0x40
 [<ffffffff811a046d>] writeback_single_inode+0xdd/0x2c0
 [<ffffffff811a08ae>] writeback_sb_inodes+0xce/0x180
 [<ffffffff811a0a0b>] writeback_inodes_wb+0xab/0x1b0
 [<ffffffff811a0dab>] wb_writeback+0x29b/0x3f0
 [<ffffffff814eca40>] ? thread_return+0x4e/0x77e
 [<ffffffff8107cc02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811a1099>] wb_do_writeback+0x199/0x240
 [<ffffffff811a11a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff81090ab7>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81134dc6>] bdi_start_fn+0x86/0x100
 [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81090886>] kthread+0x96/0xa0
 [<ffffffff8100c14a>] child_rip+0xa/0x20
 [<ffffffff810907f0>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20
INFO: task jbd2/dm-5-8:10717 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
jbd2/dm-5-8   D 0000000000000006     0 10717      2 0x00000080
 ffff880391099d20 0000000000000046 0000000000015fc0 0000000000015fc0
 ffff88002c28fda0 0000000000015fc0 0000000000015fc0 ffff880420cd2b00
 ffff880420cd30b8 ffff880391099fd8 000000000000f4e8 ffff880420cd30b8
Call Trace:
 [<ffffffff81090ede>] ? prepare_to_wait+0x4e/0x80
 [<ffffffffa016a80f>] jbd2_journal_commit_transaction+0x19f/0x14b0 [jbd2]
 [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
 [<ffffffff8107c0ec>] ? lock_timer_base+0x3c/0x70
 [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa0170928>] kjournald2+0xb8/0x220 [jbd2]
 [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa0170870>] ? kjournald2+0x0/0x220 [jbd2]
 [<ffffffff81090886>] kthread+0x96/0xa0
 [<ffffffff8100c14a>] child_rip+0xa/0x20
 [<ffffffff810907f0>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20
INFO: task flush-253:5:32096 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
flush-253:5   D 0000000000000000     0 32096      2 0x00000080
 ffff8803b1e338d0 0000000000000046 0000000000000000 ffffffff81012b59
 ffff8803b1e33870 ffffffff8109b809 ffff8803a0911038 0000000000000286
 ffff8803a0911038 ffff8803b1e33fd8 000000000000f4e8 ffff8803a0911038
Call Trace:
 [<ffffffff81012b59>] ? read_tsc+0x9/0x20
 [<ffffffff8109b809>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff81110b10>] ? sync_page+0x0/0x50
 [<ffffffff814ed1e3>] io_schedule+0x73/0xc0
 [<ffffffff81110b4d>] sync_page+0x3d/0x50
 [<ffffffff814eda4a>] __wait_on_bit_lock+0x5a/0xc0
 [<ffffffff81110ae7>] __lock_page+0x67/0x70
 [<ffffffff81090c30>] ? wake_bit_function+0x0/0x50
 [<ffffffff811271a5>] ? pagevec_lookup_tag+0x25/0x40
 [<ffffffffa018aa16>] ext4_num_dirty_pages+0x256/0x260 [ext4]
 [<ffffffffa01689f6>] ? jbd2_journal_stop+0x1e6/0x2b0 [jbd2]
 [<ffffffffa018f1d4>] ext4_da_writepages+0x594/0x660 [ext4]
 [<ffffffff8126a5e9>] ? cpumask_next_and+0x29/0x50
 [<ffffffff81054754>] ? find_busiest_group+0x244/0xb20
 [<ffffffff81126351>] do_writepages+0x21/0x40
 [<ffffffff811a046d>] writeback_single_inode+0xdd/0x2c0
 [<ffffffff811a08ae>] writeback_sb_inodes+0xce/0x180
 [<ffffffff811a0a0b>] writeback_inodes_wb+0xab/0x1b0
 [<ffffffff811a0dab>] wb_writeback+0x29b/0x3f0
 [<ffffffff814eca40>] ? thread_return+0x4e/0x77e
 [<ffffffff8107cc02>] ? del_timer_sync+0x22/0x30
 [<ffffffff811a1099>] wb_do_writeback+0x199/0x240
 [<ffffffff811a11a3>] bdi_writeback_task+0x63/0x1b0
 [<ffffffff81090ab7>] ? bit_waitqueue+0x17/0xd0
 [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81134dc6>] bdi_start_fn+0x86/0x100
 [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100
 [<ffffffff81090886>] kthread+0x96/0xa0
 [<ffffffff8100c14a>] child_rip+0xa/0x20
 [<ffffffff810907f0>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20
INFO: task jbd2/dm-5-8:10717 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
jbd2/dm-5-8   D 0000000000000006     0 10717      2 0x00000080
 ffff880391099d20 0000000000000046 0000000000015fc0 0000000000015fc0
 ffff88002c24fda0 0000000000015fc0 0000000000015fc0 ffff880420cd2b00
 ffff880420cd30b8 ffff880391099fd8 000000000000f4e8 ffff880420cd30b8
Call Trace:
 [<ffffffff81090ede>] ? prepare_to_wait+0x4e/0x80
 [<ffffffffa016a80f>] jbd2_journal_commit_transaction+0x19f/0x14b0 [jbd2]
 [<ffffffff810096f0>] ? __switch_to+0xd0/0x320
 [<ffffffff81059c99>] ? find_busiest_queue+0x69/0x150
 [<ffffffff8107c0ec>] ? lock_timer_base+0x3c/0x70
 [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa0170928>] kjournald2+0xb8/0x220 [jbd2]
 [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40
 [<ffffffffa0170870>] ? kjournald2+0x0/0x220 [jbd2]
 [<ffffffff81090886>] kthread+0x96/0xa0
 [<ffffffff8100c14a>] child_rip+0xa/0x20
 [<ffffffff810907f0>] ? kthread+0x0/0xa0
 [<ffffffff8100c140>] ? child_rip+0x0/0x20



2013/5/21 Mia Lueng <xiaozunvlg at gmail.com>

> drbd configure :
>
> [root at kvm3 ~]# drbdadm dump drbd0
> # resource drbd0 on kvm3: not ignored, not stacked
> # defined at /etc/drbd.d/drbd0.res:1
> resource drbd0 {
>     on kvm3 {
>         device           /dev/drbd0 minor 0;
>         disk             /dev/vg_kvm3/drbd0;
>         meta-disk        internal;
>         address          ipv4 192.168.10.6:7700;
>     }
>     on kvm4 {
>         device           /dev/drbd0 minor 0;
>         disk             /dev/vg_kvm4/drbd0;
>         meta-disk        internal;
>         address          ipv4 192.168.10.7:7700;
>     }
>     net {
>         protocol           A;
>         csums-alg        md5;
>         verify-alg       md5;
>         ping-timeout      30;
>         ping-int          30;
>         max-epoch-size   8192;
>         max-buffers      8912;
>         unplug-watermark 131072;
>     }
>     disk {
>         on-io-error      pass_on;
>         disk-barrier      no;
>         disk-flushes      no;
>         resync-rate      100M;
>         c-plan-ahead      20;
>         c-delay-target   100;
>         c-max-rate       400M;
>         c-min-rate        2M;
>         al-extents       601;
>     }
> }
>
>
> I run iozone on drbd and it blocks for a long time.
>
> /root/work/iozone/iozone3_414/src/current/iozone -a -i 0 -i 1 -Rb
> /root/drbd_100M_test.xls
>
> # cat /proc/drbd
> version: 8.4.3 (api:1/proto:86-101)
> GIT-hash: 2acb4389deadd704f072570faf373c835c028bb1 build by root at kvm3,
> 2013-05-08 23:19:24
>  0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r---n-
>     ns:29990400 nr:0 dw:29995024 dr:1393 al:1441 bm:0 lo:0 pe:1156 ua:0
> ap:1156 ep:1 wo:d oos:0
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130523/fa4f932b/attachment.htm>


More information about the drbd-user mailing list