Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Of course, I have completely forgotten to mention the most important assumption, that the file replication has to be asynchronous! I understand all the implecations of a synchronous file replication over WAN, which basically is useless in our situation, since it has to be a high performance solution on the local sites. I thought that DRBD was capable of doing this asynchronous. We will look into the csync2 project and make a test setup. Prior we have discarded rsync and other solutions, because it has to be real-time or close to real-time file replication. I know that this is a very needed solution and that many project based companies have problems with this, but I still have not found or have been presented a workable solution for linux. We are def. willing to pay for it, but it has to be a proven solution and nothing experimental. If it is stable and does the above we are willing to test it and pay for it. thanks Kristian Lars Ellenberg skrev: > / 2007-03-05 17:07:06 +0100 > \ Kristian T Knudsen: >> Hello, >> >> We have been waiting for years to have a setup that can replicate our >> project files between our sites through 2-4Mbit dsl (max 30sek. delay). > > so for synchronous (drbd protocol C) or "quasi synchronous" (drbd protocol A) > replication, you'd have sustained write throughput of max 200 - 500 kByte/sec > you should have an application that would not really write too much. > >> I read that version 0.8x would have the feature that you can mount the >> disks from both sites and thus write to the disks from both sites at the >> same time and the changes would be replicated back and forth. > > yes, when you use a cluster file system. I'd consider it "experimental" > to use cluster file systems over WAN, though. > >> We are looking for the simplest setup first, 2 node active-passive >> cluster (sles 10, heartbeat) and a DAS disk system connected to both >> serveres. This setup on 2 different locations with 500km between them >> (Tinglev, Denmark and Berlin, Germany). We have a dedictated E1(~2mbit) >> line with about 25ms delay for the data transmission. The disks are >> about 500GB and the data generated each day is about 500MB, but the data >> change is a lot more we assume serveral GB/day. Also noteworhty is that >> we have millions small files (10-50kb). > > some numbers: > normal local scsi disk: > latency ~ 2 to 3 ms, throughput ~ 100 MegaByte per second > good local raid system with battery backed write buffer and stuff: > sub millisecond, 200 MegaByte per second and more > > LAN, GigE: rtt ~ 0.2 ms, ~ 100 MegaByte/second > > your line: 25ms rtt, ~ 200 KiloByte / second > > so, if you run drbd on top of your line, > you'd have about whatever your local readperformance would be. > but your write performance would feel like one of those > first generation CD writers. > > the maximum you could expect for a 2Mbit line when it transfers at its > limits 24 hours: about 16 GByte per day. > >> Later we would connect a third site to this setup and replicate >> between them. Perhaps in the future also use ocfs2 to make it >> active-active on each site with the 2 node clusters or do I actually >> need to make use of the clsuter file system now to be able to it in >> the first place? > > you should use OCFS2 right away, if you intend to somewhen use it > active/active. But given the circumstances, I won't recommend this. > one network glitch, and ocfs will do a self-fence :-/ > >> The main part is that the files are kopied between sites and replace >> older versions and that when you open a file on one site, that you on >> another site can se it is opened from someone with the fil-locking >> feature from dedicated applications such as AutoCAD, which uses .DWL >> "(DraWing Lock) files are temporary lock files created when a drawing >> file (.DWG) is opened." >> >> So the obvious question ... is that possible with 8.0.1, and has someone >> by any change a similar setup that has it working? > > it would be possible. > but my understanding of AutoCad and similar, is, that it produces and > handles potentially large files, changes some small things in it, > and then writes it all back to disk. > > what you'd probably need is some asynchronous file based rsync-like > replication triggered by file system events. > if you contact us at LinBit, we might be able to tailor something for > you with the semantics you need based on our csync2 ... >