Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
> -----Original Message----- > From: drbd-user-bounces at lists.linbit.com > [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of > Kristian Knudsen > Sent: Thursday, March 08, 2007 5:19 AM > To: drbd-user at lists.linbit.com > Subject: [DRBD-user] Re: file replication between sites > > > Of course, I have completely forgotten to mention the most important > assumption, that the file replication has to be asynchronous! > > I understand all the implecations of a synchronous file > replication over > WAN, which basically is useless in our situation, since it > has to be a > high performance solution on the local sites. I thought that DRBD was > capable of doing this asynchronous. It is through Prot A, but you will most likely still have issues with a) total amount of data replicated in 24 hours, b) current drbd only supports 2 replicas, and c) most cluster file systems expect to have synchronous data access between nodes. > We will look into the csync2 project and make a test setup. Prior we > have discarded rsync and other solutions, because it has to > be real-time > or close to real-time file replication. > > I know that this is a very needed solution and that many > project based > companies have problems with this, but I still have not found or have > been presented a workable solution for linux. We are def. > willing to pay > for it, but it has to be a proven solution and nothing > experimental. If > it is stable and does the above we are willing to test it and > pay for it. > > thanks > Kristian The best solution may not be replication of storage, but the centralization of the applications. By having users either log on completely through terminal services, citrix, remote X to a centralized app server that has local access to the storage over the WAN or have just those applications exported over Citrix, X to the users' desktop over the WAN where they will read and write and execute remotely but display locally. You will then need only 2 storage units, either as primary or backup or as primary-primary with load balancing between the two application servers, if you can find a clustering file system that can work asynchronously and that the application will work properly with, or for disaster recovery, offsite backup, snapshot server, etc. I would probably go with the application server solution connecting to the storage via fiber channel or iSCSI to a locally based storage server, depending on the cost benefit of either, and have drbd do asynchronous replication to a backup site off the storage server connected with a dedicated E1 for data replication, or a committed rate of E1 equivalence for replication if using larger pipes. Try to keep the amount of replication to a minimum for reliability, manageability, and performance otherwise it gets too complicated to diagnose, tune and fix problems. > Lars Ellenberg skrev: > > / 2007-03-05 17:07:06 +0100 > > \ Kristian T Knudsen: > >> Hello, > >> > >> We have been waiting for years to have a setup that can > replicate our > >> project files between our sites through 2-4Mbit dsl (max > 30sek. delay). > > > > so for synchronous (drbd protocol C) or "quasi synchronous" > (drbd protocol A) > > replication, you'd have sustained write throughput of max > 200 - 500 kByte/sec > > you should have an application that would not really write too much. > > > >> I read that version 0.8x would have the feature that you > can mount the > >> disks from both sites and thus write to the disks from > both sites at the > >> same time and the changes would be replicated back and forth. > > > > yes, when you use a cluster file system. I'd consider it > "experimental" > > to use cluster file systems over WAN, though. > > > >> We are looking for the simplest setup first, 2 node active-passive > >> cluster (sles 10, heartbeat) and a DAS disk system > connected to both > >> serveres. This setup on 2 different locations with 500km > between them > >> (Tinglev, Denmark and Berlin, Germany). We have a > dedictated E1(~2mbit) > >> line with about 25ms delay for the data transmission. The disks are > >> about 500GB and the data generated each day is about > 500MB, but the data > >> change is a lot more we assume serveral GB/day. Also > noteworhty is that > >> we have millions small files (10-50kb). > > > > some numbers: > > normal local scsi disk: > > latency ~ 2 to 3 ms, throughput ~ 100 MegaByte per second > > good local raid system with battery backed write buffer and stuff: > > sub millisecond, 200 MegaByte per second and more > > > > LAN, GigE: rtt ~ 0.2 ms, ~ 100 MegaByte/second > > > > your line: 25ms rtt, ~ 200 KiloByte / second > > > > so, if you run drbd on top of your line, > > you'd have about whatever your local readperformance would be. > > but your write performance would feel like one of those > > first generation CD writers. > > > > the maximum you could expect for a 2Mbit line when it > transfers at its > > limits 24 hours: about 16 GByte per day. > > > >> Later we would connect a third site to this setup and replicate > >> between them. Perhaps in the future also use ocfs2 to make it > >> active-active on each site with the 2 node clusters or do > I actually > >> need to make use of the clsuter file system now to be able to it in > >> the first place? > > > > you should use OCFS2 right away, if you intend to somewhen use it > > active/active. But given the circumstances, I won't recommend this. > > one network glitch, and ocfs will do a self-fence :-/ > > > >> The main part is that the files are kopied between sites > and replace > >> older versions and that when you open a file on one site, > that you on > >> another site can se it is opened from someone with the fil-locking > >> feature from dedicated applications such as AutoCAD, which > uses .DWL > >> "(DraWing Lock) files are temporary lock files created > when a drawing > >> file (.DWG) is opened." > >> > >> So the obvious question ... is that possible with 8.0.1, > and has someone > >> by any change a similar setup that has it working? > > > > it would be possible. > > but my understanding of AutoCad and similar, is, that it > produces and > > handles potentially large files, changes some small things in it, > > and then writes it all back to disk. > > > > what you'd probably need is some asynchronous file based rsync-like > > replication triggered by file system events. > > if you contact us at LinBit, we might be able to tailor > something for > > you with the semantics you need based on our csync2 ... > > > > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user > ______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.