[Csync2] csync2 prefix mapping ==> I/O Error 'No such file or directory' in rsync-check

[Lars Ellenberg]

On Mon, Jun 18, 2012 at 09:58:57PM +0530, Samba wrote:
> Noticed an issue with master-master setup
> 
> Configured Auto : younger
> 
> modified a file under subdir (C) A/B/C
> 
> then after sometime deleted the subdir (C) A/B/C
> 
> csync2 -x from deletion side:
> 
> errored with "Do not auto-resolve conflict; this is removal"
> 
> Ideally, the subdir "C" should have been deleted on the other peer.
> 
> Interestingly, when i run "csyc2 -x" from the other peer [since this is
> master-master configuration], the file modification got synced back(with a
> message 'auto resolving conflict : won younger/older test') to the node
> where the subdir has been deleted.
> 
> Same is true with deletion happening on the peer and modification on the
> local; it seems the modification is getting synced and deletion is failing
> irrespective of whichever is younger, although csync2 is configured to sync
> the 'younger' changes. If that part is corrected, then i  guess even the
> deletions would sync properly.

How do you determine that the deletion was younger?

There is no stat information on a non-existing file,
and there is no deletion time stamp.

If we have a conflict of deletion against modification,
modification wins. I think that was a concious decision.

Of course, if you want to solve that cleanly for the special case
that the recorded (in the csync2 db) modification time on the
deletion side is already more recent than the current modification time
of the peer, go ahead.

But that means that you did a modification,
then check (to record it in the db),
but NOT sync that change, and only later deleted,
while doing a modification on the other side as well.

I don't think there is a "clean" solution that always fits.

> 
> Thanks and Regards,
> Samba
> ====================================================
> 
> On Mon, Jun 18, 2012 at 7:12 PM, Samba <saasira at gmail.com> wrote:
> 
> > Lars,
> >
> > An important fact that need to be understood from these tests is that even
> > after this fix, csync2 is not able to sync unmodified content to those
> > directories that were deleted on the slave server; it is only able to sync
> > just the modified content.

Don't do that, then.

 ;-)

Also, there is csync2 -m -r
and there is csync2 -TUXI

> > A quick try could be to mark any parent folders [up in the hierarchy] as
> > dirty on the slave but i doubt if that could trigger the master to sync the
> > whole directory directory tree path; an ideal fix should be send a response
> > to the master [via conn_printf] that the directory tree starting from so
> > and so subdirectory has to be marked dirty and resynced.

Does not feel right.

> > I'm not sure how to introduce that new 'tag' in the csync2 communication
> > protocol; it would be great if you can give some pointer.

If you think you need a new command,
in daemon.c there is cmdtab[], used by csync_daemon_session().

If you think you can get away with a special "reply",
there is read_conn_status(), which is line based,
and currently only checks for no line (connection_closed_error),
 "OK ("  arbitrary text ... (whatever the command was, it succeeded)
 and anything else, where
 "File is also marked dirty here!" is a special string.

You could introduce a new special string,
and have read_conn_status() return some enum,
maybe OK, ERROR, ALSO_MARKED_DIRTY, PARENT_DIR_MISSING ?
(it is now hardcoded 0, 1, 2)

see all those very ugly
if (last_conn_status = read_conn_status()) goto maybe_auto_resolve;
in update.c

> >> Lars,
> >>
> >> I did not make that change myself but I felt that it would be safe to
> >> resolve the prefixes in get_tempfile_name() and paranoid_file() functions
> >> rather than expecting that the caller functions would resolve the prefixes
> >> before passing the input parameters.
> >>
> >> Centralizing the logic has the other benefits like :
> >>
> >>    - resolving environment variables
> >>    - resolving symbolic links
> >>    - resolving relative paths, etc
> >>
> >> Regards,
> >> Samba
> >>
> >>
> >> ==================================================================================
> >>
> >> On Mon, Jun 18, 2012 at 2:27 PM, Samba <saasira at gmail.com> wrote:
> >>
> >>> Hi Lars,
> >>>
> >>> I managed to modify the patch submitted by *Dennis Schafroth *to also
> >>> consider the configured 'tempdir' directive in csync2 config files.
> >>> Further, Dennis's patch did not fix the issue of replication to deleted
> >>> subdirectories on slave server, which this patch has fixed. On the
> >>> otherhand, this patch also includes the fix given by Dennis for backing up
> >>> of prefixed locations.
> >>>
> >>>
> >>> The attached patch fixes the issue previously mentioned in this mail
> >>> thread, namely 'fail to sync when a subdirectory gets deleted either on
> >>> master or slave servers, WITH OR WITHOUT PREFIX MAPPING'.
> >>>
> >>>
> >>> I tested it for regression and am confident that this did not introduce
> >>> any new issues;
> >>>
> >>> Here are the test cases we have run after upgrading the Csync2 rpm

Cool. Many thanks.
> >>> dirtree deleted on master
> >>>
> >>> some files in the hierarchy changed on slave

Did you also test
  dirtree deleted on one, some files added on the other?
That usually leeds to a pathological situation...



> >>>  Please review the attached patch and give feedback [ this is my first
> >>> attempt at writing any usable C program, so please suggest appropriate
> >>> corrections if any].
> >>>
> >>> I hope this patch gets committed to trunk soon.

Will take a few days. Please bump if you don't hear back by next week.

Thanks again.


	Lars