From bill at webmail.us  Wed Sep 13 14:10:39 2006
From: bill at webmail.us (Bill Boebel)
Date: Wed Sep 13 14:10:42 2006
Subject: [Csync2] Syncing files that get renamed frequently
Message-ID: <48103.10.238.10.70.1158149439.webmail@10.238.10.70>

Is csync2 a good solution for syncing a large number of files that get renamed frequently?

For example, I have a pair of servers that I'd like to keep in sync.  There are 1.2 million files that each range from 4k to 25 MB in size; 150 GB total.  About 100,000 of these files get renamed daily.  Does csync2 perform a delete/copy for each rename, or is it capable of replicating the rename action?

I browsed the archives and documentation and could not find any information about how file rename's are replicated.

Thanks,
Bill

From paul at mjr.org  Fri Sep 22 09:32:41 2006
From: paul at mjr.org (Paul Hedderly)
Date: Fri Sep 22 09:32:52 2006
Subject: [Csync2] CSYNC2 through NAT / masquerdes
Message-ID: <1158910361.21879.16.camel@lh>

Thankyou Linbit for a bit of software that could be a lifesaver... if I
can make it work properly for me :O)

I am running the Debian sid/etch CSYNC2 on several machines (Debian
version 1.33-2).

On machines that are on the same subnet it works.

But I would like to run it across a VPN link to another site, and would
really like it to run on laptops that are often VPN connected.

When it is attempted to run over the VPN I get:

tawny# csync2 -Tvvv
My hostname is tawny.xxx.priv.
Database-File: /var/lib/csync2/tawny.xxx.priv.db
Config-File:   /etc/csync2.cfg
Running in-sync check for 172.16.1.25 <-> 172.16.48.25.
Connecting to host 172.16.48.25 (SSL) ...
Local> SSL\n
Peer> OK (activating_ssl).\n
Establishing SSL connection failed.

and at the other end:

pallas# csync2 -Tvvv
My hostname is tawny.xxx.priv.
Database-File: /var/lib/csync2/tawny.xxx.priv.db
Config-File:   /etc/csync2.cfg
Running in-sync check for 172.16.1.25 <-> 172.16.48.25.
Connecting to host 172.16.48.25 (SSL) ...
Local> SSL\n
Peer> OK (activating_ssl).\n
Establishing SSL connection failed.

And with SSL disabled:

tawny:/etc# csync2 -xv
Connecting to host pallas.xxx.priv (PLAIN) ...
While syncing file /data/home/prh/.Xauthority:
ERROR from peer pallas.xxx.priv: Connection closed.
While syncing file /etc/csync2.cfg:
ERROR from peer pallas.xxx.priv: Connection closed.
ERROR from peer pallas.xxx.priv: Connection closed.
Finished with 3 errors.

and:

pallas:/data/home# csync2 -xv
Connecting to host tawny.xxx.priv (PLAIN) ...
While syncing file /etc/csync2.cfg:
ERROR from peer tawny.xxx.priv: Connection closed.
ERROR from peer tawny.xxx.priv: Connection closed.
Finished with 2 errors.

The IP addresses/hostnames are fine. The problem seems to be that the
route the packets take from one end to the other takes them through at
least one sNAT/masquerade so the packets get to their destination
correctly, but the other end does not recognise the source IP address.

I've tried with and without SSL and just cannot make it work. Any hints?

(I can provide straces and pcap files if they would help.)

-- 
Paul Hedderly <paul@mjr.org>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.linbit.com/pipermail/csync2/attachments/20060922/4815ca44/attachment.pgp
From Lars.Ellenberg at linbit.com  Fri Sep 22 10:44:01 2006
From: Lars.Ellenberg at linbit.com (Lars Ellenberg)
Date: Fri Sep 22 10:44:00 2006
Subject: [Csync2] CSYNC2 through NAT / masquerdes
In-Reply-To: <1158910361.21879.16.camel@lh>
References: <1158910361.21879.16.camel@lh>
Message-ID: <20060922084401.GB7496@soda.linbit>

/ 2006-09-22 07:32:41 +0000
\ Paul Hedderly:
> Thankyou Linbit for a bit of software that could be a lifesaver... if I
> can make it work properly for me :O)
> 
> I am running the Debian sid/etch CSYNC2 on several machines (Debian
> version 1.33-2).
> 
> On machines that are on the same subnet it works.
> 
> But I would like to run it across a VPN link to another site, and would
> really like it to run on laptops that are often VPN connected.

> The IP addresses/hostnames are fine. The problem seems to be that the
> route the packets take from one end to the other takes them through at
> least one sNAT/masquerade so the packets get to their destination
> correctly, but the other end does not recognise the source IP address.


you could run the csync2 daemon in debug mode, like
 csync2 -iii -vvv
to see what the daemon side tells you.

I guess it says "Identification failed!".

csync2 does a forward lookup (gethostbyname()) on the hostname provided
by the hello, and compares that to the ip found with getpeername on the
socket. if they don't match, sorry.

you'll find the corresponding code in daemon.c, case A_HELLO: .

maybe you can use the "somehostname@real-host-name" syntax in the csync2
config file. maybe you need to tweak your /etc/hosts file, so it would
provide the "expected" ip for the forward lookup.

if you figure it out, for the benefit of others, please post a
description of your setup, the problem, and your solution.

cheers,

-- 
: Lars Ellenberg                                  Tel +43-1-8178292-0  :
: LINBIT Information Technologies GmbH            Fax +43-1-8178292-82 :
: Schoenbrunner Str. 244, A-1120 Vienna/Europe   http://www.linbit.com :
