Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hello,
On Mon, 3 Mar 2008 09:34:30 +0100 Florian Haas wrote:
> And you are perfectly certain you know what you're doing in setting
> use-bmbv?
>
No, but a question from me about this last Thursday went unanswered so far
and the documentation suggests that it speeds things up.
So for the sake of determining the most effective setup...
> And you are also perfectly certain you want to set max-buffers !=
> max-epoch-size?
>
Nothing I read suggested that they need to equal, but changing both to
2048 made no difference. Not that I expected any, given that a test with
protocol B gave pretty much the same results as C.
> And you have a good reason to disable the sndbuf-size 512k you had
> already set?
>
Yeah, since it did not have any noticeable effect on the performance.
It did test more than just the config posted, but the results did not vary
considerably.
Right now I'm doing a run with use-bmbv off, max buffers and epoch at 8192
and sndbuf at 1MB. ethstats hovers around:
---
eth1: 446.41 Mb/s In 6.09 Mb/s Out - 9123.4 p/s In 4778.3 p/s Out
---
Which suggest the results will be slightly better, but still it's just
using half of the available bandwidth.
What is the most throughput anybody has achieved on writes with Protocol
C and what settings (and network hardware) did they use?
> Just my €.02.
>
Just my 2yen. ^_^
Christian
P.S.
The results just came back:
---
Version 1.03 ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
borg00a 50000M 54917 17 47675 15 410592 55 392.4 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
128 79514 95 +++++ +++ 65768 76 78165 94 +++++ +++ 81634 100
---
So yes, slightly better, but still nothing to write home about.
Which of the changes above you think brought this about and will cranking
up things to even more insane levels get me to about 80MB/s (which is what
I could settle for)?
>
> On Saturday 01 March 2008 06:49:30 Christian Balzer wrote:
> > Hello,
> >
> > I'm trying to build a HA cluster here. Each node has 8 2.66GHz cpu
> > cores, 24GB RAM and 8 1TB SATA drives behind a LSI (Fusion MPT) SAS
> > 1068E controller. Interconnection is via one of 4 1GE interfaces,
> > directly. Kernel is 2.6.22.18 and DRBD is 8.0.11, the storage device
> > in question is a 3TB MD RAID5 spread across all 8 drives. The native
> > results for this device using ext3 and bonnie for benchmarking are:
> > ---
> > Version 1.03 ------Sequential Output------ --Sequential Input-
> > --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec
> > %CP /sec %CP borg00a 50000M 120486 36 87998
> > 17 535665 44 390.9 1 ------Sequential Create------
> > --------Random Create-------- -Create-- --Read--- -Delete-- -Create--
> > --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec
> > %CP /sec %CP 128 74265 90 +++++ +++ 83659 100 71540 88 +++++ +++
> > 81619 99 ---
> >
> > The same test done on the resulting (UpToDate) drbd device:
> > ---
> > Version 1.03 ------Sequential Output------ --Sequential Input-
> > --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
> > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec
> > %CP /sec %CP borg00a 50000M 41801 13 39659
> > 11 413367 37 397.7 1 ------Sequential Create------
> > --------Random Create-------- -Create-- --Read--- -Delete-- -Create--
> > --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec
> > %CP /sec %CP 128 78847 95 +++++ +++ 86936 99 78722 95 +++++ +++
> > 63054 76 ---
>
--
Christian Balzer Network/Systems Engineer NOC
chibi at gol.com Global OnLine Japan/Fusion Network Services
http://www.gol.com/