Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi, I have a fairly simple goal for which DRBD seems a perfect choice. Only thing is, I will need an additional component to get where I want to be. After looking up and down the Internet I am now at the point where there are too many options of which I cannot envision the impact/complexity so I am hoping for some advice from people who have been there. I have two application servers that need to do failover and load balancing. On both servers I need read/write access to the same local files so processing on a certain file can restart on the other server in case of failover. Additionally, I would like file-locking (flock and/or lockf) to work on the shared files. I think it boils down to these choices: * dual-primary setup of DRBD with some clustered file system (GFS maybe), * primary-secondary setup of DRBD on my database servers with NFS, * primary-secondary setup of DRBD with Lustre, * use MySQL Cluster instead of DRBD. The first option is discouraged for production purposes on drbd.org (I forgot where, but it is mentioned). Using NFS with DRBD will probably only work if I run it on the database servers (we two of those as well) because running an NFS client on the same machine as the server is discouraged (see http://www.linux-ha.org/HaNFS). I don't really like having the files not local even on the master application server. Since I have no actual experience with this kind of set up I am worried that failover will not always be transparent to client processes. Using Lustre with DRBD seems a good fit for my situation, but I wonder if it's overkill for our situation and if it's going to be complex to deploy. Since we are going to use MySQL Cluster for storing meta information for the tasks we need to process, we might store our files in the database as well. As with NFS this has the drawback that I will be storing files on a different node. The files we process for a task can grow either large (in excess of 100MB) or plentiful (more than 100.000 files) on a regular basis, so I am not thrilled of storing all of that in a database when a local file is what I need. For sake of simplicity I am leaning toward NFS with DRBD, but Lustre seems to fit my wishes better. Or should I simply go for primary-primary? Any alternative suggestions are also welcome. I looked into all kinds of stuff but that does not seem to offer any better options: * clustered LVM is intended for something entirely different I think * pNFS is not usable and maybe also not intended for my purpose * other clustered file systems (GlusterFS, GPFS,...?) would probably still need something like dual-primary DRBD Any advice on the direction to take would be great. This is our architecture: * 2 application servers and 2 database servers * all servers run Debian5 * the application is written in Java, based on the Spring framework and runs in Tomcat * we use ActiveMQ for JMS messaging and this should also help us achieve failover and load balancing * MySQL Cluster is used for application persistence and will be set up with two data nodes on our database servers, and two management nodes together with two MySQL front-ends on the application servers. File locking on the shared storage would allow ActiveMQ to have automatic fail-back. (http://activemq.apache.org/shared-file-system-master-slave.html) Below is some more background info on what we are doing. We are developing an application that will process potentially large files (PDF and PostScript files that can contain upward of 100.000 pages, which can amount to hundreds of MBs per file) or more than 100.000 separate files (some 20KB each). The intention is to achieve failover and load balancing using 2 application servers and 2 database servers. For the database technology we have chosen MySQL Cluster which seems to be able to achieve load balancing and failover out of the box if we use our two database servers as data nodes and let the application servers host MySQL front-ends and management nodes. The database will be used to store meta information about the tasks we are processing. Because of the potential size of our content files I don't want to store these in the database. For the application we intent to rely on failover and load balancing capabilities of ActveMQ once we are processing a task. Submission of tasks will mainly occur by polling hotfolders. I don't think we will have much trouble implementing the hotfolder in a master/slave setup and let heartbeat (or some clever JMS messaging) take care of failover. Processing a content file is triggered by a JMS message, so as long as ActiveMQ does its job I don't really need to worry about both servers processing the same file. So that leaves me with one issue: we need to store my content files in a way that both application servers have read/write access to them. When an application server fails, we depend on ActiveMQ to resend a JMS message on the survivor which will then have to restart processing. So the only thing needed is that the same files are available. If you read up to here, thanks for your patience ;) Thanks, Manuel.