Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
# sysctl -a|grep dirty
vm.dirty_background_ratio = 10
vm.dirty_background_bytes = 0
vm.dirty_ratio = 20
vm.dirty_bytes = 0
vm.dirty_writeback_centisecs = 500
vm.dirty_expire_centisecs = 3000
bandwidth is 100M bps
2013/5/9 Lars Ellenberg <lars.ellenberg at linbit.com>
> On Thu, May 09, 2013 at 12:16:56AM +0800, Mia Lueng wrote:
> > in drbd 8.4.3，I do the following test:
> >
> > [root at kvm3 drbd.d]# drbdadm dump drbd0
> > # resource drbd0 on kvm3: not ignored, not stacked
> > # defined at /etc/drbd.d/drbd0.res:1
> > resource drbd0 {
> > on kvm3 {
> > device /dev/drbd0 minor 0;
> > disk /dev/vg_kvm3/drbd0;
> > meta-disk internal;
> > address ipv4 192.168.10.6:7700;
> > }
> > on kvm4 {
> > device /dev/drbd0 minor 0;
> > disk /dev/vg_kvm4/drbd0;
> > meta-disk internal;
> > address ipv4 192.168.10.7:7700;
> > }
> > net {
> > protocol A;
> > csums-alg md5;
> > verify-alg md5;
> > ping-timeout 30;
> > ping-int 30;
> > max-epoch-size 8192;
> > max-buffers 8912;
> > unplug-watermark 131072;
> > }
> > disk {
> > on-io-error pass_on;
> > disk-barrier no;
> > disk-flushes no;
> > resync-rate 100M;
> > c-plan-ahead 20;
> > c-delay-target 100;
> > c-max-rate 400M;
> > c-min-rate 2M;
> > al-extents 601;
> > }
> > }
> >
> > [root at kvm3 oradata]# dd if=t1 of=t2 bs=1M
> > 5585+1 records in
> > 5585+1 records out
> > 5856305152 bytes (5.9 GB) copied, 286.119 s, 20.5 MB/s
>
> That writes to the page cache, and from there to the block device.
>
> No fsync, no sync: there will still be a few GB in the cache (RAM only).
>
> > [root at kvm3 oradata]# cd
> > [root at kvm3 ~]# umount /oradata
> >
> >
> > it takes lots of time(up to 600 seconds) to umount the drbd mount point.
>
> On umount, the filesystem obviously has to flush all dirty pages first.
>
> What is your replication bandwidth?
>
> > echo "1" >/proc/sys/vm/block_dump
> > show when umount ,
> >
> > [root at kvm3 ~]# dmesg|tail -n 100
> ...
> > umount(3958): WRITE block 100925440 on dm-5
> > umount(3958): WRITE block 100925440 on dm-5
> > umount(3958): WRITE block 100925440 on dm-5
> > umount(3958): WRITE block 0 on dm-5
> > umount(3958): dirtied inode 1053911 (mtab.tmp) on dm-0
> > umount(3958): dirtied inode 1053911 (mtab.tmp) on dm-0
> > umount(3958): WRITE block 33845632 on dm-0
> > umount(3958): dirtied inode 1053912 (?) on dm-0
> >
> >
> > Is the reason that I use protocol A?
>
> No.
>
> But that you need to understand caching, and tunables.
>
> Some hints and keywords for a followup search:
>
> Check how much "dirty" data (writes not yet on stable storage)
> is still in RAM:
> grep Dirty /proc/meminfo
>
> Tune how much dirty data is "allowed"
> sysctl
> vm.dirty_background_bytes
> vm.dirty_bytes
> vm.dirty_writeback_centisecs
> vm.dirty_expire_centisecs
>
> also compare:
> time dd if=t1 of=t2 bs=1M; time sync
> time dd if=t1 of=t2 bs=1M conv=fsync
>
>
>
>
> --
> : Lars Ellenberg
> : LINBIT | Your Way to High Availability
> : DRBD/HA support and consulting http://www.linbit.com
>
> DRBD(R) and LINBIT(R) are registered trademarks of LINBIT, Austria.
> __
> please don't Cc me, but send to list -- I'm subscribed
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130509/5219db6c/attachment.htm>