<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: Times New Roman; font-size: 12pt; color: #000000'><font class="Apple-style-span" size="3" face="'andale mono', times">Here are the stats results with Bonnie++, in HTML format (see attachment)</font><div><font class="Apple-style-span" face="'andale mono', times">The first 4 tests are in VM.</font></div><div><font class="Apple-style-span" face="'andale mono', times">The 2 last are just test on 2 workstations with similar options. An entry level one and a high end one, both with SSDs. DRBD is of course not used there.<br></font><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">Test command :</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">-------------------</font></div><div><font class="Apple-style-span" face="'andale mono', times">bonnie++ -d ./scratchdir/ -c 2 -s 2G -n 64:4096:512 (Changed to -s 16G on my high end workstation )</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">Virtualization software and Hardware for the first 4 tests (localssd1, ssdvg1, localdir, satavg1) :</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">------------------------------------------------------------------------------------------------------------------------</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- Virtulization : Proxmox VE 1.9 (KVM) + VirtIO network & disk drivers, 10GB Storage and 256MB Ram</font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- Hardware : 2 HP Pavilion G5415be workstation with QuadCore Athlon X4 645 3.2GHz+ SSD Chronos SA3 MSK 240GB ( in SATA2 mode because of Mother board ) - Limited to 1GB of Ram with the mem=1024M boot option. </font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- Additional GBits PCIe 1X network adapters with a cross cable dedicated to DRBD replication (when used).</font></div></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">Test conditions (rest of Stack description ):</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">localssd1</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">=======</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM OS : Ubuntu 11.10 Server with default options (ext4)</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM Storage : LVM on (SSD PV of 40GB)</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">ssdvg1</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">=====</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM OS : Ubuntu 11.10 Server with default options (ext4) with 256MB Ram </font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM Storage : LVM on DRBD Active-Active (Protocol C) (SSD PV of 200GB in each server)</font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"></font></div></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">localdir</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">=====</font></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM OS : Ubuntu 11.10 Server with default options (ext4) with 256MB Ram </font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM Storage : RAW Image on a local SATA disk (2TB/7200RPM) formated in EXT3</font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div></div><div style="color: rgb(0, 0, 0); font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">satavg1</font></div><div><font class="Apple-style-span" face="'andale mono', times">=====</font></div><div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM OS : Ubuntu 11.10 Server with default options (ext4) with 256MB Ram </font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times">- VM Storage : LVM on DRBD Active-Active (Protocol C) (SATA disk 2TB/7200RPM PV of 750GB in each server) - With Meta-Disk on a small partition of 128MB on the </font></div><div style="font-size: 12pt; "><font class="Apple-style-span" face="'andale mono', times"><br></font></div></div><div><div><font class="Apple-style-span" face="'andale mono', times"><br>skinner</font></div><div><font class="Apple-style-span" face="'andale mono', times">=====</font></div><div><font class="Apple-style-span" face="'andale mono', times">- Not a VM, just my workstation, a very light workstation with 2GB Ram, Dual Core ATOM 1.8GHz, and OCZ Vertex2 SSD.</font></div><div><font class="Apple-style-span" face="'andale mono', times">- Kubuntu 11.10 on ext4</font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times">westvleteren</font></div><div><font class="Apple-style-span" face="'andale mono', times">=========</font></div><div><font class="Apple-style-span" face="'andale mono', times">- Not a VM, an other workstation, high end, with 8GB Ram, Quad Core intel Corei5, SSD 120GB SATA3 ( more than 500MB/s read/writes with 90000 IOPS )</font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times">Conclusions : </font></div><div><font class="Apple-style-span" face="'andale mono', times">-----------------</font></div><div><font class="Apple-style-span" face="'andale mono', times">- on the satavg1 test, using meta-disk on SSD did not fix my problem. Maybe it's better than meta-disk internal, but clearly unusable. Machine unresponsive, host with load of 10 during the test in the VM, ... So unfortunately, SSD meta-disk did not save my 750GB partitions from being far too slow to be used.</font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times">- on the </font><span class="Apple-style-span" style="font-family: 'andale mono', times; font-size: 16px; ">ssdvg1, the performances, are really good. Strangely, it even competes to the results without DRBD (localssd1). The move to SSD for my VMs seems to fix my issue. My totally useless cluster on SATA Drives, becomes now really usable with SSD.</span></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times">As all my VMs can keep into 200GB storage. My performance problem is fixed, it's even really good.</font></div><div><font class="Apple-style-span" face="'andale mono', times"><br></font></div><div><font class="Apple-style-span" face="'andale mono', times">Regards,<br><br>François<br><br><br><br><br><br><br><br></font><br><br><br><br><br><font class="Apple-style-span" size="3">I noticed I did not reply to the list. So, here is the copy : </font><br><br><br><font class="Apple-style-span" size="3">Hi Eduardo, </font><br><br><br><font class="Apple-style-span" size="3">When I had these horrible performances I was not in a good situations : </font><br><font class="Apple-style-span" size="3">- One 2GB disk SSD 7200RPM per node. </font><br><font class="Apple-style-span" size="3">- DRBD 8.3, Protocol C, Active-Active </font><br><font class="Apple-style-span" size="3">- 2 Network interfaces on a 100Mbits network switch. </font><br><br><br><font class="Apple-style-span" size="3">Now that I added the SSDs, I'll test again. I'm currently looking on how to test. </font><br><font class="Apple-style-span" size="3">As I'm not interested in testing my OS cache, I'll reduce the memory to 1GB or less (with the mem=1024M kernel boot option), Install an ubuntu VM with 512MRam and PV disk drivers, and run either bonnie++ or iozone in these 4 conditions : </font><br><font class="Apple-style-span" size="3">- Local SSD </font><br><font class="Apple-style-span" size="3">- DRBD SSD in Active-Active </font><br><font class="Apple-style-span" size="3">- Local SATA </font><br><font class="Apple-style-span" size="3">- DRBD Sata in Active-Active (but with the meta-disk on SSD). (Where I was getting 30IOPS with bonnie++ in my VM before I moved meta-disk to SSD) </font><br><br><br><font class="Apple-style-span" size="3">Regards, </font><br><font class="Apple-style-span" size="3">François Delpierre </font><br><hr id="zwchr" style="color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 12pt; "><br><font class="Apple-style-span" size="3">De: "Eduardo Diaz - Gmail" <ediazrod@gmail.com> </font><br><font class="Apple-style-span" size="3">À: "François Delpierre" <francois.delpierre@pivert.org> </font><br><font class="Apple-style-span" size="3">Cc: drbd-user@lists.linbit.com </font><br><font class="Apple-style-span" size="3">Envoyé: Mardi 3 Janvier 2012 14:19:39 </font><br><font class="Apple-style-span" size="3">Objet: Re: [DRBD-user] drbd performance issue on SATA </font><br><br><font class="Apple-style-span" size="3">I check the Network link too, because some times the network are the </font><br><font class="Apple-style-span" size="3">real problem. </font><br><br><font class="Apple-style-span" size="3">Did you test the network interface?. </font><br><br><font class="Apple-style-span" size="3">Did you change the the protocol to A for example.. and make more test. </font><br><br><font class="Apple-style-span" size="3">In my experience the SSD disk has problems for broken. </font><br><br><font class="Apple-style-span" size="3">http://www.codinghorror.com/blog/2011/05/the-hot-crazy-solid-state-drive-scale.html </font><br><br><font class="Apple-style-span" size="3">What configuration are you creating? Active/active, Active/passive? </font><br><font class="Apple-style-span" size="3">What filesystem? </font><br><br><font class="Apple-style-span" size="3">regards! </font><br><br><font class="Apple-style-span" size="3">On Sun, Jan 1, 2012 at 6:04 PM, François Delpierre </font><br><font class="Apple-style-span" size="3"><francois.delpierre@pivert.org> wrote: </font><br><blockquote style="color: rgb(0, 0, 0); font-family: 'Times New Roman'; font-size: 12pt; "> Hi, <br> <br> I'm building a very low cost virtualization server for a Non Profit <br> organization using 2 HP workstations (500€ each) (8GB Ram, 1.5TB, QuadCore <br> AMD), Software Proxmox and DRBD for the data replication with dedicated <br> gigabit & cross cable for the replication. <br> <br> After setting up the replication (Protocol C), I had catastrophic <br> performances, ( dropping to 30 IOPS after 60s of random writes ). <br> <br> I also found a lot of reading, not fully explaining my terrific 30 IOPS on <br> 7200RPM drives, but clearly pointing the necessity to have a battery backed <br> up cache controller to sustain random writes generated by the meta-disk. <br> <br> As the hardware makrket did change quite a lot recently, especially towards <br> SSD, I immediately realized that 240GB SSD are half price than a small raid <br> controller, a battery for the cache, and 2 small disks with reasonable <br> performances (10rpm). So, I choose SSD. <br> <br> I created 2 partitions on my SSD : <br> - 128MB for external meta-disk of my slow SATA partition (750GB) <br> - 240GB for a second DRBD resource, with internal meta-disk. <br> <br> My questions : <br> - Is there some recommendations on a SSD setup ? <br> - Why is the external meta-disk that few documented ? I spent more than 1 <br> hour to find the syntax of the directive : <br> <br> meta-disk <br> /dev/disk/by-id/scsi-SATA_MKNSSDCR240GB_MKN1146A0000032XXX-part1[0]; <br> <br> And it's not clear to my what is the [0] at the end ? <br> <br> Regards, <br> <br> François Delpierre <br> <br> <br> _______________________________________________ <br> drbd-user mailing list <br> drbd-user@lists.linbit.com <br> http://lists.linbit.com/mailman/listinfo/drbd-user <br> <br></blockquote></div></div></div></div></body></html>