Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On Thu, Aug 25, 2005 at 12:46:30AM +0930, David wrote: > > Thanks paddy for your reply. I think I am better understanding the > differences Glad to help. Hope I've not mislead you :) > - I am not after the full-scale mysql clustering solution; I > don't need it/have the resources. I'd love to know what its for. > What do you mean when you say you have mysql running on top of drbd? Have > you got both PC's mysql data directories separate, or do they share the same > data files? (ie. /mnt/drbd0/var/lib/mysql) yes. I have an ext3 filesystem which I mount on /var/lib/mysql. It is mounted on the primary side and my haresources looks like: node1 drbddisk::r0 Filesystem::/dev/drbd0::/var/lib/mysql::ext3 mysql node2 (removing some extraneous noise for clarity) heartbeat takes care of the dependencies, there is a drbd init.d script but neither the filesystem mount nor mysql are started by the regular init. The resources.di/mysql script is really no more than: op=$1; case $op in start) /etc/init.d/mysql start;; stop) /etc/init.d/mysql stop;; status|monitor) if mysqld_status check_alive nowarn; then echo "running" else echo "stopped" fi ;; *) usage exit 1;; esac exit $? where mysqld_status() is just cribbed from the code in the debian init.d script. The debian init.d script seems to handle the mysql fsck :) I've cobbled together a nasty trick with bind to fail over a dns name, so that a writer can access the db from the secondary node. I've seen mysql fsck and complain a couple of times so far, without any apparent harm :) Not seen any problem at the filesystem level yet (nor do I expect to, it seems to be well tested). For my application (simply logging - unrelated inserts a row at a time) its even possible for the whole setup to go split brain (as it has done a couple of times) with minimal pain to me. I just dump the extra records on one side, resync and then load the dump in. It's so absurd that I'm almost ashamed to admit it, but it does what I want. I'm also fortunate that there are various secondary systems which would enable me to rebuild the database without loss if it all went horribly wrong. Not exactly your average database problem ;) To be honest, I'm not even watching it terribly closely right now. It could be going wrong and I just haven't noticed :) (actually, I'm aware of bugs elsewhere, so it's not just a question of whether it goes wrong, but what goes wrong and how often, which is why its more dificult for me to be certain that there isn't a problem with the mysql over drbd part of iti. That and I'm working on other things :). > I was wondering if such a setup as the latter is possible, since it seems > logical to me and pretty straight-forward; if a server running mysql > crashes, it must reboot and recover. If the primary server of a two-server > heartbeat "cluster" crashes, same thing - it's just a different machine > which must recover from the cluster, it is not privy to any extra/different > information... or is it? This is pretty much my reasoning too. Its not too much of a stretch to imagine that drbd might introduce a reordering of writes that could perhaps violate assumptions in the filesystem fsck or in the mysql fsck, or even just tickle unused codepaths and shake out interesting bugs, even if drbd is correct. Its clear that the authors have taken considerable care about this aspect of drbd, but did I read something on this list about fscking a journalled filesystem here recently ? There is also the issue of whether a commit propagates correctly. My assumption would be that mysql would return from a commit only after some reasonably conservative effort has been made to write the necessary data to disk and that drbd protocol A will thus guarantee that the commit is written both sides. It is an assumption based entirely on imagination, without foundation in research. By comparison, my best understanding of mysql replication was that a commit could get lost if the master crashed before it propagated to a slave. (I could be wrong on this. I'd be happy to be corrected). Other possible pros and cons include: There may be some efficiencies to be had by doing the replication in mysql where there is more knowledge of the data. I don't actually care about this. Something like drbd protocol A is likely to add latency to certain operations. Even if performance is not a consideration, this may challenge the assumptions that clients make about timeouts, etc. It is probably easier to find support and experience for the built-in facilities of mysql. Even with the possibility of losing a commit, mysql style replication should give a better chance that you'll at least have something running. I've seen too many filesystems go south over the years to imagine that the risk that the secondary can't automatically fsck the failed-over block device, or the same but for mysql, is zero. Clearly its not, and a having a shared resource may be less robust for certain applications. Actually this is one of the reasons I plan to look at mysql replication long term: for me, losing a commit temporarily is better than losing availability. But right now admin time is more precious :) By a similar token, drbd doesn't know about the applications above it. If it replicates data in good faith, but the data is bad, there is no filtering at the secondary node to check that the data is good from an application point of view. Conversely having a single well tested service rather than having every application reinvent the wheel can help with the bug count. The bottom line is that, as far as I know, this setup is essentially untested, and all hope of it working are based purely on my fevered imagining :) Since I like the look of drbd for other problems, and this problem is non-critcal for me, this is a freebie for me, otherwise I couldn't jusitfy it. Looked at another way, its a chance for me to try out and get to know drbd before commiting more deeply. So far drbd does well. > Does anybody see any danger with this proposed setup? yes :) > Anybody actually doing this? yes :) > What do you mean by mysql on drbd - could you say (briefly) how you have > mysql running on drbd (ie. which files are available to which servers?) I hope I covered this above. Feel free to enquire further on any details. Sorry, I missed the bit earlier about having multiple secondaries in drbd. As you know, I had assumed that this is not possible with 0.7, but I don't recall exactly why. I'm not in a position to try it out right now, but I am interested to hear whether it works. Regards, Paddy -- Perl 6 will give you the big knob. -- Larry Wall