Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Christian Balzer wrote: > Hello, > > On Mon, 3 Mar 2008 09:34:30 +0100 Florian Haas wrote: > >> And you are perfectly certain you know what you're doing in setting >> use-bmbv? >> >> > No, but a question from me about this last Thursday went unanswered so far > and the documentation suggests that it speeds things up. > So for the sake of determining the most effective setup... > > >> And you are also perfectly certain you want to set max-buffers != >> max-epoch-size? >> >> > Nothing I read suggested that they need to equal, but changing both to > 2048 made no difference. Not that I expected any, given that a test with > protocol B gave pretty much the same results as C. > > >> And you have a good reason to disable the sndbuf-size 512k you had >> already set? >> >> > Yeah, since it did not have any noticeable effect on the performance. > > It did test more than just the config posted, but the results did not vary > considerably. > Right now I'm doing a run with use-bmbv off, max buffers and epoch at 8192 > and sndbuf at 1MB. ethstats hovers around: > --- > eth1: 446.41 Mb/s In 6.09 Mb/s Out - 9123.4 p/s In 4778.3 p/s Out > --- > Which suggest the results will be slightly better, but still it's just > using half of the available bandwidth. > What is the most throughput anybody has achieved on writes with Protocol > C and what settings (and network hardware) did they use? > I have 4 NIC Intel Ethernet cluster like you, and i take out of it 115MBytes write. The controller behind it gives me about 160MBytes writing, but I get the Gbit Ethernet limit, i have 4 Gbit Ethernet bonding but i cant get out that, even in network tests, because of the load balancing algorithm used by the switch. I never manage to get on 4Gbit Ethernet trunk, with one switch in the middle, for the same mac address. For this you have to buy a switch that does the same algorithm that Linux bonding driver does (mode 0), or LACP with the same algorithm. If anyone knows one, please tell me, i would like to test it too. For different mac address's in network tests i get about 3.3Gbit, without tuning (MTU). Good luck. > >> Just my €.02. >> >> > Just my 2yen. ^_^ > > Christian > > P.S. > The results just came back: > --- > Version 1.03 ------Sequential Output------ --Sequential Input- --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > borg00a 50000M 54917 17 47675 15 410592 55 392.4 1 > ------Sequential Create------ --------Random Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 128 79514 95 +++++ +++ 65768 76 78165 94 +++++ +++ 81634 100 > --- > So yes, slightly better, but still nothing to write home about. > Which of the changes above you think brought this about and will cranking > up things to even more insane levels get me to about 80MB/s (which is what > I could settle for)? > > >> On Saturday 01 March 2008 06:49:30 Christian Balzer wrote: >> >>> Hello, >>> >>> I'm trying to build a HA cluster here. Each node has 8 2.66GHz cpu >>> cores, 24GB RAM and 8 1TB SATA drives behind a LSI (Fusion MPT) SAS >>> 1068E controller. Interconnection is via one of 4 1GE interfaces, >>> directly. Kernel is 2.6.22.18 and DRBD is 8.0.11, the storage device >>> in question is a 3TB MD RAID5 spread across all 8 drives. The native >>> results for this device using ext3 and bonnie for benchmarking are: >>> --- >>> Version 1.03 ------Sequential Output------ --Sequential Input- >>> --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >>> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec >>> %CP /sec %CP borg00a 50000M 120486 36 87998 >>> 17 535665 44 390.9 1 ------Sequential Create------ >>> --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- >>> --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec >>> %CP /sec %CP 128 74265 90 +++++ +++ 83659 100 71540 88 +++++ +++ >>> 81619 99 --- >>> >>> The same test done on the resulting (UpToDate) drbd device: >>> --- >>> Version 1.03 ------Sequential Output------ --Sequential Input- >>> --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- >>> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec >>> %CP /sec %CP borg00a 50000M 41801 13 39659 >>> 11 413367 37 397.7 1 ------Sequential Create------ >>> --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- >>> --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec >>> %CP /sec %CP 128 78847 95 +++++ +++ 86936 99 78722 95 +++++ +++ >>> 63054 76 --- >>> > > > -- Igor Neves <igor.neves at 3gnt.net> 3GNTW - Tecnologias de Informação, Lda SIP: igor at 3gnt.net JID: igor at 3gnt.net ICQ: 249075444 MSN: igor at 3gnt.net TLM: 00351914503611 PSTN: 00351252377120