<div dir="ltr">there are some error messages in dmesg:<div><br></div><div><div>INFO: task flush-253:5:32096 blocked for more than 120 seconds.</div><div>"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.</div>
<div>flush-253:5 D 0000000000000002 0 32096 2 0x00000080</div><div> ffff8803b1e339b0 0000000000000046 0000000000000000 ffff88002c292920</div><div> ffff8803b1e33960 ffffffff81127e77 0000000300000001 0000000000000002</div>
<div> ffff8803a0911038 ffff8803b1e33fd8 000000000000f4e8 ffff8803a0911038</div><div>Call Trace:</div><div> [<ffffffff81127e77>] ? ____pagevec_lru_add+0x167/0x180</div><div> [<ffffffffa0168f6a>] start_this_handle+0x25a/0x4d0 [jbd2]</div>
<div> [<ffffffff8115e5db>] ? cache_alloc_refill+0x15b/0x240</div><div> [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40</div><div> [<ffffffffa01693c5>] jbd2_journal_start+0xb5/0x100 [jbd2]</div><div>
[<ffffffffa018b035>] ? ext4_meta_trans_blocks+0x75/0xf0 [ext4]</div><div> [<ffffffffa01a5218>] ext4_journal_start_sb+0x58/0x90 [ext4]</div><div> [<ffffffffa018eebc>] ext4_da_writepages+0x27c/0x660 [ext4]</div>
<div> [<ffffffff81054754>] ? find_busiest_group+0x244/0xb20</div><div> [<ffffffff81126351>] do_writepages+0x21/0x40</div><div> [<ffffffff811a046d>] writeback_single_inode+0xdd/0x2c0</div><div> [<ffffffff811a08ae>] writeback_sb_inodes+0xce/0x180</div>
<div> [<ffffffff811a0a0b>] writeback_inodes_wb+0xab/0x1b0</div><div> [<ffffffff811a0dab>] wb_writeback+0x29b/0x3f0</div><div> [<ffffffff814eca40>] ? thread_return+0x4e/0x77e</div><div> [<ffffffff8107cc02>] ? del_timer_sync+0x22/0x30</div>
<div> [<ffffffff811a1099>] wb_do_writeback+0x199/0x240</div><div> [<ffffffff811a11a3>] bdi_writeback_task+0x63/0x1b0</div><div> [<ffffffff81090ab7>] ? bit_waitqueue+0x17/0xd0</div><div> [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100</div>
<div> [<ffffffff81134dc6>] bdi_start_fn+0x86/0x100</div><div> [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100</div><div> [<ffffffff81090886>] kthread+0x96/0xa0</div><div> [<ffffffff8100c14a>] child_rip+0xa/0x20</div>
<div> [<ffffffff810907f0>] ? kthread+0x0/0xa0</div><div> [<ffffffff8100c140>] ? child_rip+0x0/0x20</div><div>INFO: task jbd2/dm-5-8:10717 blocked for more than 120 seconds.</div><div>"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.</div>
<div>jbd2/dm-5-8 D 0000000000000006 0 10717 2 0x00000080</div><div> ffff880391099d20 0000000000000046 0000000000015fc0 0000000000015fc0</div><div> ffff88002c28fda0 0000000000015fc0 0000000000015fc0 ffff880420cd2b00</div>
<div> ffff880420cd30b8 ffff880391099fd8 000000000000f4e8 ffff880420cd30b8</div><div>Call Trace:</div><div> [<ffffffff81090ede>] ? prepare_to_wait+0x4e/0x80</div><div> [<ffffffffa016a80f>] jbd2_journal_commit_transaction+0x19f/0x14b0 [jbd2]</div>
<div> [<ffffffff810096f0>] ? __switch_to+0xd0/0x320</div><div> [<ffffffff8107c0ec>] ? lock_timer_base+0x3c/0x70</div><div> [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40</div><div> [<ffffffffa0170928>] kjournald2+0xb8/0x220 [jbd2]</div>
<div> [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40</div><div> [<ffffffffa0170870>] ? kjournald2+0x0/0x220 [jbd2]</div><div> [<ffffffff81090886>] kthread+0x96/0xa0</div><div> [<ffffffff8100c14a>] child_rip+0xa/0x20</div>
<div> [<ffffffff810907f0>] ? kthread+0x0/0xa0</div><div> [<ffffffff8100c140>] ? child_rip+0x0/0x20</div><div>INFO: task flush-253:5:32096 blocked for more than 120 seconds.</div><div>"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.</div>
<div>flush-253:5 D 0000000000000000 0 32096 2 0x00000080</div><div> ffff8803b1e338d0 0000000000000046 0000000000000000 ffffffff81012b59</div><div> ffff8803b1e33870 ffffffff8109b809 ffff8803a0911038 0000000000000286</div>
<div> ffff8803a0911038 ffff8803b1e33fd8 000000000000f4e8 ffff8803a0911038</div><div>Call Trace:</div><div> [<ffffffff81012b59>] ? read_tsc+0x9/0x20</div><div> [<ffffffff8109b809>] ? ktime_get_ts+0xa9/0xe0</div>
<div> [<ffffffff81110b10>] ? sync_page+0x0/0x50</div><div> [<ffffffff814ed1e3>] io_schedule+0x73/0xc0</div><div> [<ffffffff81110b4d>] sync_page+0x3d/0x50</div><div> [<ffffffff814eda4a>] __wait_on_bit_lock+0x5a/0xc0</div>
<div> [<ffffffff81110ae7>] __lock_page+0x67/0x70</div><div> [<ffffffff81090c30>] ? wake_bit_function+0x0/0x50</div><div> [<ffffffff811271a5>] ? pagevec_lookup_tag+0x25/0x40</div><div> [<ffffffffa018aa16>] ext4_num_dirty_pages+0x256/0x260 [ext4]</div>
<div> [<ffffffffa01689f6>] ? jbd2_journal_stop+0x1e6/0x2b0 [jbd2]</div><div> [<ffffffffa018f1d4>] ext4_da_writepages+0x594/0x660 [ext4]</div><div> [<ffffffff8126a5e9>] ? cpumask_next_and+0x29/0x50</div><div>
[<ffffffff81054754>] ? find_busiest_group+0x244/0xb20</div><div> [<ffffffff81126351>] do_writepages+0x21/0x40</div><div> [<ffffffff811a046d>] writeback_single_inode+0xdd/0x2c0</div><div> [<ffffffff811a08ae>] writeback_sb_inodes+0xce/0x180</div>
<div> [<ffffffff811a0a0b>] writeback_inodes_wb+0xab/0x1b0</div><div> [<ffffffff811a0dab>] wb_writeback+0x29b/0x3f0</div><div> [<ffffffff814eca40>] ? thread_return+0x4e/0x77e</div><div> [<ffffffff8107cc02>] ? del_timer_sync+0x22/0x30</div>
<div> [<ffffffff811a1099>] wb_do_writeback+0x199/0x240</div><div> [<ffffffff811a11a3>] bdi_writeback_task+0x63/0x1b0</div><div> [<ffffffff81090ab7>] ? bit_waitqueue+0x17/0xd0</div><div> [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100</div>
<div> [<ffffffff81134dc6>] bdi_start_fn+0x86/0x100</div><div> [<ffffffff81134d40>] ? bdi_start_fn+0x0/0x100</div><div> [<ffffffff81090886>] kthread+0x96/0xa0</div><div> [<ffffffff8100c14a>] child_rip+0xa/0x20</div>
<div> [<ffffffff810907f0>] ? kthread+0x0/0xa0</div><div> [<ffffffff8100c140>] ? child_rip+0x0/0x20</div><div>INFO: task jbd2/dm-5-8:10717 blocked for more than 120 seconds.</div><div>"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.</div>
<div>jbd2/dm-5-8 D 0000000000000006 0 10717 2 0x00000080</div><div> ffff880391099d20 0000000000000046 0000000000015fc0 0000000000015fc0</div><div> ffff88002c24fda0 0000000000015fc0 0000000000015fc0 ffff880420cd2b00</div>
<div> ffff880420cd30b8 ffff880391099fd8 000000000000f4e8 ffff880420cd30b8</div><div>Call Trace:</div><div> [<ffffffff81090ede>] ? prepare_to_wait+0x4e/0x80</div><div> [<ffffffffa016a80f>] jbd2_journal_commit_transaction+0x19f/0x14b0 [jbd2]</div>
<div> [<ffffffff810096f0>] ? __switch_to+0xd0/0x320</div><div> [<ffffffff81059c99>] ? find_busiest_queue+0x69/0x150</div><div> [<ffffffff8107c0ec>] ? lock_timer_base+0x3c/0x70</div><div> [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40</div>
<div> [<ffffffffa0170928>] kjournald2+0xb8/0x220 [jbd2]</div><div> [<ffffffff81090bf0>] ? autoremove_wake_function+0x0/0x40</div><div> [<ffffffffa0170870>] ? kjournald2+0x0/0x220 [jbd2]</div><div> [<ffffffff81090886>] kthread+0x96/0xa0</div>
<div> [<ffffffff8100c14a>] child_rip+0xa/0x20</div><div> [<ffffffff810907f0>] ? kthread+0x0/0xa0</div><div> [<ffffffff8100c140>] ? child_rip+0x0/0x20</div></div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">2013/5/21 Mia Lueng <span dir="ltr"><<a href="mailto:xiaozunvlg@gmail.com" target="_blank">xiaozunvlg@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">drbd configure :<div><br></div><div><div>[root@kvm3 ~]# drbdadm dump drbd0</div><div># resource drbd0 on kvm3: not ignored, not stacked</div><div># defined at /etc/drbd.d/drbd0.res:1</div><div>resource drbd0 {</div>
<div> on kvm3 {</div><div> device /dev/drbd0 minor 0;</div><div> disk /dev/vg_kvm3/drbd0;</div><div> meta-disk internal;</div><div> address ipv4 <a href="http://192.168.10.6:7700" target="_blank">192.168.10.6:7700</a>;</div>
<div> }</div><div> on kvm4 {</div><div> device /dev/drbd0 minor 0;</div><div> disk /dev/vg_kvm4/drbd0;</div><div> meta-disk internal;</div><div> address ipv4 <a href="http://192.168.10.7:7700" target="_blank">192.168.10.7:7700</a>;</div>
<div> }</div><div> net {</div><div> protocol A;</div><div> csums-alg md5;</div><div> verify-alg md5;</div><div> ping-timeout 30;</div><div> ping-int 30;</div>
<div> max-epoch-size 8192;</div><div> max-buffers 8912;</div><div> unplug-watermark 131072;</div><div> }</div><div> disk {</div><div> on-io-error pass_on;</div><div> disk-barrier no;</div>
<div> disk-flushes no;</div><div> resync-rate 100M;</div><div> c-plan-ahead 20;</div><div> c-delay-target 100;</div><div> c-max-rate 400M;</div><div> c-min-rate 2M;</div>
<div> al-extents 601;</div><div> }</div><div>}</div></div><div><br></div><div><br></div><div>I run iozone on drbd and it blocks for a long time.</div><div><br></div><div>/root/work/iozone/iozone3_414/src/current/iozone -a -i 0 -i 1 -Rb /root/drbd_100M_test.xls<br>
</div><div><br></div><div><div># cat /proc/drbd</div><div>version: 8.4.3 (api:1/proto:86-101)</div><div>GIT-hash: 2acb4389deadd704f072570faf373c835c028bb1 build by root@kvm3, 2013-05-08 23:19:24</div><div> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate A r---n-</div>
<div> ns:29990400 nr:0 dw:29995024 dr:1393 al:1441 bm:0 lo:0 pe:1156 ua:0 ap:1156 ep:1 wo:d oos:0</div><div><br></div><div><br></div></div><div><br></div><div><br></div></div>
</blockquote></div><br></div>