Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
> how much memory do you have? 1GIG ECC DDR > did you try to watch some other monitoring tool during these tests? > e.g. "vmstat 1"? Wasn't sure what I was looking for when reading the output... > did you check the write cache (hdparm)? root at mailbox2:/src/hdparm-6.9# hdparm -i /dev/sdb /dev/sdb: HDIO_GET_IDENTITY failed: Inappropriate ioctl for device root at mailbox2:/src/hdparm-6.9# hdparm -I /dev/sdb /dev/sdb: HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device I don't think the RAID controller wants to give it (hdparm) access. > does something change when you use > ./dm -a 0 -b 1M -s 3G -y -m -p -x -o /dev/sdb > ^^^ > (-x: use O_DIRECT...) ./dm -a 0 -b 1M -s 3G -y -m -p -o /dev/sdb ~ consistent 70 MB/s ./dm -a 0 -b 1M -s 3G -x -y -m -p -o /dev/sdb ~ consistent 50 MB/s > > > ((you should not use /dev/sdb when that is configured as lower level > device for drbd)) ok, as per your instructions, I tried this and here are the results... running this "./dm -a 0 -b 1M -s 3G -y -m -p -x -o /dev/sdb" concurrently produces the same repetitive results, and those are about a 70 MB/s benchmark. now I "modprobe drbd" and "drbdadm adjust all" and "drbdadm primary all" (I only have one resource BTW and state is disconnected) and ... running this "./dm -a 0 -b 1M -s 3G -y -m -p -o /dev/drbd0" concurrently produces the same repetitive results, and those are about a 33 MB/s benchmark. Constantly 33MB/s nothing higher, nothing lower. I am losing more that 50% just by going through the DRBD block device when it's disconnected from the peer! > > == drbd disconnected == > > when you write to drbd, and it is disconnected, > drbd needs to update its activity log and dirty bitmap every now and > then, which introduces additional writes, and seeks to the appropriate > drbd meta data locations and back... I'm using an internal meta disk, would that cause the more than 50% reduction?