[Csync2] Sync triggering best practices (includes a small script)..

[Jamie Tufnell]

Hi,

Firstly, thank you for this fantastic tool!

I've recently set up csync2 on one of our smaller clusters and I'm
wondering about best practices when it comes to deciding when to run
csync2 -x.

Do you run it periodically from cron?
Do you use filesystem notification tools (like inotifywait in Linux)?
Do you just run it manually when you make changes?

I can think of pros/cons with every method depending on the
environment and data turnover.

With manual syncing there's always the chance of forgetting a sync...
especially if you are doing a lot of syncs, or haven't done one in a
while.

A cronjob every 5 minutes could potentially break the slaves for 5
minutes if it occurs in the middle of an important multi-file sync.

A simple inotify-based solution would end up calling csync2 -x many
many times if the master is being updated heavily.  I've seen scripts
like this pasted on the web in a few places.  There's one half way
down this page: http://pooteeweet.org/blog/909/p/2

I decided to extend that approach so that csync2 isn't called until 20
seconds after the last filesystem event in that batch.  I can of
course, think of cons of this approach too, but I think it should be
OK in our environment.

I'm really interested to hear more experienced people's feedback on
this approach (and implementation!), as well as how you handle this
problem in your environment.

I've included the source below which is in PHP (company policy)
formatted for 80 cols:

#!/usr/bin/php
<?php # vim: syntax=php:ts=4:et:

$inotifywait_path = "/tmp";
$inotifywait_cmd = "/usr/bin/inotifywait -mrq -e close_write -e create
-e delete \
    -e move $inotifywait_path";

$threshold = 20; // seconds to wait after the last fs event before syncing

$p = popen($inotifywait_cmd, "r");
stream_set_blocking($p, false);

$dirty = false;
while (true) {
    // clear any pending inotifywait(1) output
    do {
        if (false === ($buf = fread($p, 8192))) {
            echo "fatal: read error\n";
            exit;
        }
        $bytes_read = strlen($buf);
        if ($bytes_read > 0) {
            $dirty = true;
            $last_dirtied = time();
            echo "$buf\n";
        }
    } while ($bytes_read > 0);

    // if we need to sync, display message with estimated sync time,
    // and/or perform the sync.
    if ($dirty) {
        if ((time() - $last_dirtied) >= $threshold) {
            echo "*** syncing now..\n";
            passthru($csync_cmd, $retval);
            if ($retval != 0) {
                echo "fatal: $csync2 had errors\n";
                exit;
            }
            $dirty = false;
        } else {
            $eta = $threshold - (time() - $last_dirtied);
            echo "notice: last event at $last_dirtied, syncing in ${eta}s..\n";
        }
    }

    sleep(1);
}

Thanks!
J.