[DRBD-user] Significant performance improvement!

Lee Christie Lee at titaninternet.co.uk
Sat Jun 21 00:03:02 CEST 2008


The recommended practice for critical servers is to disable the write
cache on the hard drives, use a good raid controller with a good sized
cache and battery backup unit.
 
Hard disks these days have lots of cache (32MB in some cases), so in the
event of a poweroff you stand to lose a lot of data. Some raid cards
have the RAM and battery on a module so in the event of raid controller
failure, you can take the data held in RAM and transport to a new
controller.
 
In the tests that I did, comparing the "flushes" options, the
performance was so incredibly different that I would go so far as to be
a bit cheeky and say that something is broken in the code. Or perhaps
over-sensitive, you choose the terms to use but the bottom line is that
adding those parameters completely transforms the throughput.
 
As an aside, we found no difference in our limited performance tests
with internal or external metadata. 
 
Lee.


________________________________

	From: drbd-user-bounces at lists.linbit.com
[mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of Marcelo Azevedo
	Sent: 20 June 2008 22:37
	To: drbd-user at lists.linbit.com
	Subject: Re: [DRBD-user] Significant performance improvement!
	
	
	Dear Lars,
	 
	The backend devices are on a IBM ServeRAID-8k SAS Controller
<http://www-132.ibm.com/webapp/wcs/stores/servlet/ProductDisplay?product
Id=4611686018425211442&storeId=1&langId=-1&catalogId=-840> 
	with 256MB battery backed write cache
	 
	you are saying i should not have seen a performence hit with a 
	battery backed write cache?
	 
	i now tried  with internal metada ,  options :
	    no-disk-flushes;
	    no-md-flushes;

	 and it now works almost to naive speed.
	 
	results are similar as when i had the meta data on separate
storage but 
	still leaving  disk flushes and md flushes on. 
	 
	so i dont understand , does it request a flush from the backend
device every write request ? it sounds like synchronous write...
	 
	Also i read that if i do have a reliable battery backed write
cache 
	i can use   no-disk-flushes;  no-md-flushes;   , but i wonder if
it is only the controller (with the  battery backed write cache) that
has protection?
	 
	Don't hardrives have their own volatile write cache of their own
? 
	 
	so in a case the controller passed the data to the HD but the hd
kept it in its cache and there was a power outage i would get
corruption?
	
	
	On Fri, Jun 20, 2008 at 3:45 PM, Lars Ellenberg
<lars.ellenberg at linbit.com> wrote:
	

		On Thu, Jun 19, 2008 at 09:28:58PM +0200, Marcelo
Azevedo wrote:
		> After placing the metadata on a different spindle (HD)
, i was able to reach
		> almost close to native speed (1-2MB/s less)
		>
		> with metadata internal i was reaching tops around half
of the native speed ,
		> 37MB/s~ :
		>  physical partition -> drbd -> ext3
		>   and
		> 63MB/s~ with physical partition -> ext3
		> 61MB/s with  metadata  external, now this is true for
another strong machine .
		>
		> This other machine has hardware RAID1 with two :
		> Cheetah T10 series,
		> 146GB ,Serial Attached SCSI
		> Interface Speed: 3Gb/s
		> Spindle Rotation Speed: 15,000 RPM
		> Performance: 10K
		> on an IBM 2G Xeon server with 2 dual cpu packages ,
each cpu with 4 cores . IBM
		> ServeRAID SCSI controllers
		> 4GB of ram
		> native speed  -hardware raid1 ->  physical partition
-> ext3 is around 110MB/s
		> ( still isn't this a bit slow for this HD ? )
		> hardware raid1 -> physical partition -> drbd -> ext3 -
101MB/s with
		> external metadata on a USB2 connected SATA HD 7,200
rpm
		>
		> now this is the crazy part  - 8.3MB/s~ write speed !
with internal metadata ,
		> and 150MB/s read speed..
		> this test was repeated with bonnie , iozone and dd ,
all showed around same
		> numbers ,
		> i mean why the huge jump from 8MB/s to 100MB/s when
using external metadata ,
		> and should this be STRESSED on the Docs or when
starting the program that
		> putting the metadata on external media improves
performance significantly?  ,
		> still i don't understand why i was able to reach only
8MB/s write speed on this
		> strong server , maybe because of the hardware raid1
underneath?
		
		
	
http://www.drbd.org/users-guide/ch-internals.html#s-metadata
		 -> internal meta data
		  -> disadvantages
		
		"head movements" aka seek time.
		
		if you use internal meta data on the same single
spindle,
		without a decent battery backed write cache,
		you want to configure a large-ish al-extents,
		so drbd meta data updates happen infrequently.
		
		--
		: Lars Ellenberg
http://www.linbit.com <http://www.linbit.com/>  :
		: DRBD/HA support and consulting             sales at
linbit.com <http://linbit.com/>  :
		: LINBIT Information Technologies GmbH      Tel
+43-1-8178292-0  :
		: Vivenotgasse 48, A-1120 Vienna/Europe     Fax
+43-1-8178292-82 :
		__
		please don't Cc me, but send to list -- I'm subscribed
		_______________________________________________
		drbd-user mailing list
		drbd-user at lists.linbit.com
		http://lists.linbit.com/mailman/listinfo/drbd-user
		



-------------------------------------------------------------------------------

This email may contain legally privileged and/or confidential information. It is solely for and is confidential for use by the addressee. Unauthorised recipients must preserve, observe and respect this confidentiality. If you have received it in error please notify us and delete it from your computer. Do not discuss, distribute or otherwise copy it. 

Unless expressly stated to the contrary this e-mail is not intended to, and shall not, have any contractually binding effect on the Company and its clients. We accept no liability for any reliance placed on this e-mail other than to the intended recipient. If the content is not about the business of this Company or its clients then the message is neither from nor sanctioned by the Company. 

We accept no liability or responsibility for any changes made to this e-mail after it was sent or any viruses transmitted through this e-mail or any attachment. It is your responsibility to satisfy yourself that this e-mail or any attachment is free from viruses and can be opened without harm to your systems.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.linbit.com/pipermail/drbd-user/attachments/20080620/6b1caf97/attachment.htm 


More information about the drbd-user mailing list