[DRBD-user] Re: file replication between sites

Kristian Knudsen ktk at te.dk
Thu Mar 8 11:19:15 CET 2007

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Of course, I have completely forgotten to mention the most important 
assumption, that the file replication has to be asynchronous!

I understand all the implecations of a synchronous file replication over 
WAN, which basically is useless in our situation, since it has to be a 
high performance solution on the local sites. I thought that DRBD was 
capable of doing this asynchronous.

We will look into the csync2 project and make a test setup. Prior we 
have discarded rsync and other solutions, because it has to be real-time 
or close to real-time file replication.

I know that this is a very needed solution and that many project based 
companies have problems with this, but I still have not found or have 
been presented a workable solution for linux. We are def. willing to pay 
for it, but it has to be a proven solution and nothing experimental. If 
it is stable and does the above we are willing to test it and pay for it.

thanks
Kristian


Lars Ellenberg skrev:
> / 2007-03-05 17:07:06 +0100
> \ Kristian T Knudsen:
>> Hello,
>>
>> We have been waiting for years to have a setup that can replicate our
>> project files between our sites through 2-4Mbit dsl (max 30sek. delay).
> 
> so for synchronous (drbd protocol C) or "quasi synchronous" (drbd protocol A)
> replication, you'd have sustained write throughput of max 200 - 500 kByte/sec
> you should have an application that would not really write too much.
> 
>> I read that version 0.8x would have the feature that you can mount the
>> disks from both sites and thus write to the disks from both sites at the
>> same time and the changes would be replicated back and forth.
> 
> yes, when you use a cluster file system.  I'd consider it "experimental"
> to use cluster file systems over WAN, though.
> 
>> We are looking for the simplest setup first, 2 node active-passive
>> cluster (sles 10, heartbeat) and a DAS disk system connected to both
>> serveres. This setup on 2 different locations with 500km between them
>> (Tinglev, Denmark and Berlin, Germany). We have a dedictated E1(~2mbit)
>> line with about 25ms delay for the data transmission. The disks are
>> about 500GB and the data generated each day is about 500MB, but the data
>> change is a lot more we assume serveral GB/day. Also noteworhty is that
>> we have millions small files (10-50kb).
> 
> some numbers:
> normal local scsi disk:
>  latency ~ 2 to 3 ms, throughput ~ 100 MegaByte per second
> good local raid system with battery backed write buffer and stuff:
>  sub millisecond, 200 MegaByte per second and more
> 
> LAN, GigE: rtt ~ 0.2 ms, ~ 100 MegaByte/second
> 
> your line: 25ms rtt, ~ 200 KiloByte / second
> 
> so, if you run drbd on top of your line,
> you'd have about whatever your local readperformance would be.
> but your write performance would feel like one of those
> first generation CD writers.
> 
> the maximum you could expect for a 2Mbit line when it transfers at its
> limits 24 hours: about 16 GByte per day.
> 
>> Later we would connect a third site to this setup and replicate
>> between them. Perhaps in the future also use ocfs2 to make it
>> active-active on each site with the 2 node clusters or do I actually
>> need to make use of the clsuter file system now to be able to it in
>> the first place?
> 
> you should use OCFS2 right away, if you intend to somewhen use it
> active/active. But given the circumstances, I won't recommend this.
> one network glitch, and ocfs will do a self-fence :-/
> 
>> The main part is that the files are kopied between sites and replace
>> older versions and that when you open a file on one site, that you on
>> another site can se it is opened from someone with the fil-locking
>> feature from dedicated applications such as AutoCAD, which uses .DWL
>> "(DraWing Lock) files are temporary lock files created when a drawing
>> file (.DWG) is opened."
>>
>> So the obvious question ... is that possible with 8.0.1, and has someone
>> by any change a similar setup that has it working?
> 
> it would be possible.
> but my understanding of AutoCad and similar, is, that it produces and
> handles potentially large files, changes some small things in it,
> and then writes it all back to disk.
> 
> what you'd probably need is some asynchronous file based rsync-like
> replication triggered by file system events.
> if you contact us at LinBit, we might be able to tailor something for
> you with the semantics you need based on our csync2 ...
> 




More information about the drbd-user mailing list