Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
>From Kristian Knudsen on Thursday, March 08: > > We will look into the csync2 project and make a test setup. > Prior we have discarded rsync and other solutions, because it > has to be real-time or close to real-time file replication. csync2 actually works pretty well with this depending on how much data you have to check and/or replicate. I have a pair of webservers pretending to be accessing a "shared" filesystem which is in reality replicated by csync2 running in master-master mode. There is approximately 6GB of data (~100,000 files) which is first checked, and then synched with the other server. Each server runs the sync process via cron jobs on opposite minutes (one on odd minutes, the other on even). I do not know whether the number of files or the amount of data affects the length of time it takes for csync2 to check the filesystem, but up around 30GB of data it would take a minimum of 5 minutes to check and sync (with relatively minimal updates) between the servers... and is unacceptable in my opinion when so many of the sites are of minimal size so it's been scaled back for the time being. Currently at 6GB/100K files it takes around 45 seconds to run if there are less than 20M or so of updates. Gigabit ethernet (via crossover) and lack of SSL improves speeds greatly, but not be possible in your setup obviously. I also make a couple of time consuming rsync connections beforehand to check for active rsync/csync2 operations on the other server. This is so that new sites on one server are automatically replicated and also so csync2 does not start replicating on one server and kill off the other servers replication. With the current amount of data I have on the servers it works well for the most part with an average synchronization time of two minutes or less. I could probably run csync2 every minute on both servers and have a queue of csync2 processes waiting to run somehow but the further effort I divert to csync2 replication takes me away from the goal of getting running DRBD8 + GFS2 across the two servers. I keep running into issues with the filesystem locking up when writing to disk so I continue to have only a portion of our sites load balanced. In reality I need a bigger budget but that's another story. You could probably also run multiple instances of csync2 on a server in order to speed up replication if data can be properly segregated. You will of course have the lag between the time someone opens up a DWG file and the time the lock files get to the other server. ___________________________________________________ Dan Brown zu.com communications > I know that this is a very needed solution and that many > project based companies have problems with this, but I still > have not found or have been presented a workable solution for > linux. We are def. willing to pay for it, but it has to be a > proven solution and nothing experimental. If it is stable and > does the above we are willing to test it and pay for it. > > thanks > Kristian > > > Lars Ellenberg skrev: > > / 2007-03-05 17:07:06 +0100 > > \ Kristian T Knudsen: > >> Hello, > >> > >> We have been waiting for years to have a setup that can > replicate our > >> project files between our sites through 2-4Mbit dsl (max > 30sek. delay). > > > > so for synchronous (drbd protocol C) or "quasi synchronous" (drbd > > protocol A) replication, you'd have sustained write > throughput of max > > 200 - 500 kByte/sec you should have an application that > would not really write too much. > > > >> I read that version 0.8x would have the feature that you can mount > >> the disks from both sites and thus write to the disks from > both sites > >> at the same time and the changes would be replicated back > and forth. > > > > yes, when you use a cluster file system. I'd consider it > "experimental" > > to use cluster file systems over WAN, though. > > > >> We are looking for the simplest setup first, 2 node active-passive > >> cluster (sles 10, heartbeat) and a DAS disk system > connected to both > >> serveres. This setup on 2 different locations with 500km > between them > >> (Tinglev, Denmark and Berlin, Germany). We have a dedictated > >> E1(~2mbit) line with about 25ms delay for the data > transmission. The > >> disks are about 500GB and the data generated each day is > about 500MB, > >> but the data change is a lot more we assume serveral GB/day. Also > >> noteworhty is that we have millions small files (10-50kb). > > > > some numbers: > > normal local scsi disk: > > latency ~ 2 to 3 ms, throughput ~ 100 MegaByte per second > good local > > raid system with battery backed write buffer and stuff: > > sub millisecond, 200 MegaByte per second and more > > > > LAN, GigE: rtt ~ 0.2 ms, ~ 100 MegaByte/second > > > > your line: 25ms rtt, ~ 200 KiloByte / second > > > > so, if you run drbd on top of your line, you'd have about whatever > > your local readperformance would be. > > but your write performance would feel like one of those first > > generation CD writers. > > > > the maximum you could expect for a 2Mbit line when it > transfers at its > > limits 24 hours: about 16 GByte per day. > > > >> Later we would connect a third site to this setup and replicate > >> between them. Perhaps in the future also use ocfs2 to make it > >> active-active on each site with the 2 node clusters or do > I actually > >> need to make use of the clsuter file system now to be able > to it in > >> the first place? > > > > you should use OCFS2 right away, if you intend to somewhen use it > > active/active. But given the circumstances, I won't recommend this. > > one network glitch, and ocfs will do a self-fence :-/ > > > >> The main part is that the files are kopied between sites > and replace > >> older versions and that when you open a file on one site, > that you on > >> another site can se it is opened from someone with the fil-locking > >> feature from dedicated applications such as AutoCAD, which > uses .DWL > >> "(DraWing Lock) files are temporary lock files created > when a drawing > >> file (.DWG) is opened." > >> > >> So the obvious question ... is that possible with 8.0.1, and has > >> someone by any change a similar setup that has it working? > > > > it would be possible. > > but my understanding of AutoCad and similar, is, that it > produces and > > handles potentially large files, changes some small things > in it, and > > then writes it all back to disk. > > > > what you'd probably need is some asynchronous file based rsync-like > > replication triggered by file system events. > > if you contact us at LinBit, we might be able to tailor > something for > > you with the semantics you need based on our csync2 ... > > > > _______________________________________________ > drbd-user mailing list > drbd-user at lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user >