Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi Dan,
Thank you for pointing that out.
I didn't notice that, but the person in charge of this test said that
he did "cat /proc/drbd0" on "Secondary" node,
so he did "dd" on Primary node surely.
Anyway, I mixed some different problem
I will take a second look and repost them!
Thanks,
Junko IKEDA
2011年5月24日20:59 Dan Barker <dbarker at visioncomm.net>:
> It appears you are writing to the secondary node (Connected
> ro:Secondary/Primary). That is the source of the I/O error. Run your tests
> on a primary node.
>
> Dan
>
> -----Original Message-----
> From: drbd-user-bounces at lists.linbit.com
> [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Junko IKEDA
> Sent: Tuesday, May 24, 2011 1:01 AM
> To: drbd-user at lists.linbit.com
> Subject: Re: [DRBD-user] the perfomance issuce with a physical volume
> synchronization
>
> Hi,
>
>> "trapped in I/O error" is supposed to mean what, in this context?
>> Can you elaborate or rephrase?
>
> Here is the testing environment and logs;
>
> OS
> Red Hat Enterprise Linux Server release 6.1 (Santiago)
>
> kernel
> Linux node1 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT
> 2011 x86_64 x86_64 x86_64 GNU/Linux
>
> Hardware
> HP DL2000 (DL170e x 2）
>
> HDD
> SmartArray P410 SAS SSD 60GB x 11(RAID5)
>
> (1) ext4
> first, DRBD works well.
>
> # cat /proc/drbd
> version: 8.3.10 (api:88/proto:86-96)
> GIT-hash: 5c0b0469666682443d4785d90a2c603378f9017b build by root at node1,
> 2011-05-21 00:02:00
> 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
> ns:0 nr:11922760 dw:11922760 dr:0 al:0 bm:169 lo:0 pe:0 ua:0 ap:0
> ep:1 wo:b oos:0
>
> after mounting ext4 filesystem, run dd like this;
>
> # dd if=/dev/zero of=/mnt/test.dat bs=1M count=10000 conv=notrunc,fsync
> dd: writing `/mnt/test.dat': Read-only file system
> 7385+0 records in
> 7384+0 records out
> 7743180800 bytes (7.7 GB) copied, 37.1128 s, 209 MB/s
>
> For some reason dd gives an error.
> /proc/mounts shows the filesystem status as "ro", but mount command shows
> "rw".
>
> # cat /proc/mounts
> ...
> /dev/drbd0 /mnt ext4 ro,seclabel,relatime,barrier=1,data=ordered 0 0
>
> # mount
> ...
> /dev/drbd0 on /mnt type ext4 (rw)
>
> # tail -f /var/log/messages
> May 24 10:53:40 node1 kernel: EXT4-fs (drbd0): mounted filesystem with
> ordered data mode May 24 10:54:11 node1 kernel: JBD2: I/O error detected
> when updating journal superblock for drbd0-8.
> May 24 10:54:44 node1 kernel: Aborting journal on device drbd0-8.
> May 24 10:54:44 node1 kernel: JBD2: I/O error detected when updating journal
> superblock for drbd0-8.
> May 24 10:54:44 node1 kernel: EXT4-fs error (device drbd0):
> ext4_journal_start_sb: Detected aborted journal May 24 10:54:44 node1
> kernel: EXT4-fs (drbd0): Remounting filesystem read-only May 24 10:54:44
> node1 kernel: EXT4-fs error (device drbd0) in
> ext4_da_writepages: IO failure
> May 24 10:54:44 node1 kernel: EXT4-fs (drbd0): ext4_da_writepages:
> jbd2_start: 12288 pages, ino 12; err -30 May 24 10:54:44 node1 kernel:
>
> (2) xfs
> after mounting xfs filesystem, run dd in the same way as case (1).
>
> # dd if=/dev/zero of=/mnt/test.dat bs=1M count=10000 conv=notrunc,fsync
> dd: writing `/mnt/test.dat': Input/output error
> 4715+0 records in
> 4714+0 records out
> 4943020032 bytes (4.9 GB) copied, 12.7228 s, 389 MB/s
>
> In this case, dd gives "Input/output error".
>
> syslog said;
> May 24 11:08:32 node1 kernel: Filesystem "drbd0": Disabling barriers, trial
> barrier write failed May 24 11:08:32 node1 kernel: XFS mounting filesystem
> drbd0 May 24 11:09:02 node1 kernel: I/O error in filesystem ("drbd0")
> meta-data dev drbd0 block 0x2711172 ("xlog_iodone") error 95 buf
> count 3072
> May 24 11:09:02 node1 kernel: xfs_force_shutdown(drbd0,0x2) called from line
> 904 of file fs/xfs/xfs_log.c. Return address = 0xffffffffa038a1db May 24
> 11:09:02 node1 kernel: Filesystem "drbd0": Log I/O Error Detected. Shutting
> down filesystem: drbd0 May 24 11:09:02 node1 kernel: Please umount the
> filesystem, and rectify the problem(s) May 24 11:09:32 node1 kernel:
> Filesystem "drbd0": xfs_log_force: error
> 5 returned.
> May 24 11:10:02 node1 kernel: Filesystem "drbd0": xfs_log_force: error
> 5 returned.
> May 24 11:10:32 node1 kernel: Filesystem "drbd0": xfs_log_force: error
> 5 returned.
> May 24 11:11:02 node1 kernel: Filesystem "drbd0": xfs_log_force: error
> 5 returned.
>
> after unmounting;
> May 24 11:12:57 node1 kernel: xfs_force_shutdown(drbd0,0x1) called from line
> 1062 of file fs/xfs/linux-2.6/xfs_buf.c. Return address =
> 0xffffffffa03a6239
> May 24 11:12:57 node1 kernel: xfs_force_shutdown(drbd0,0x1) called from line
> 1062 of file fs/xfs/linux-2.6/xfs_buf.c. Return address =
> 0xffffffffa03a6239
> May 24 11:12:58 node1 kernel: Filesystem "drbd0": xfs_log_force: error
> 5 returned.
> May 24 11:12:58 node1 kernel: Filesystem "drbd0": xfs_log_force: error
> 5 returned.
>
> (3) dev/drbd0
> dd dev/drbd0 directly.
>
> # dd if=/dev/zero of=/dev/drbd0 bs=1M count=10000 conv=notrunc,fsync
> 10000+0 records in
> 10000+0 records out
> 10485760000 bytes (10 GB) copied, 93.3198 s, 112 MB/s
>
> No error.
>
> Thanks,
> Junko IKEDA
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>