Lars,<br><div style="margin-left:40px">I apologise for pushing this one a bit fast but i feel it is better to get the issue fixed in the trunk so that we do not have any PRIVATE changes in our repository.<br><br>Further, having the patch committed to the trunk will give assurance the fix is code reviewed and tested properly.<br>
<br>If you have any suggestions/corrections, then do let me know so that I can incorporate those and submit an updated patch.<br><br>You may also suggest verifying specific test cases in order to ensure better test coverage and avoid regression.<br>
<br>I really appreciate the fact that you and other commiters put extra efforts on top on your normal day-job for making this software available to users like us. I do know that it needs passion for the project and commitment for open source.<br>
<br>I hope you would find some time amidst your busy schedule to review the patch and commit it to the trunk.<br><br></div>Thanks and Regards,<br>Samba<br><div style="margin-left:40px"><br></div>PS:<br><div style="margin-left:40px">
Alternatively, how about inviting review comments from other users in the community so that we can offload the efforts to some extent. I can guess that many users would be needing a fix for the issue of 'sync failure during subdir deletion' so those folks can take this patch and review if possible or at least test and verify that it works for them. In this way we can build an community over the project and ensure that every patch is actually reviewed and tested well before getting committed to the trunk.<br>
</div><br><div style="margin-left:40px">==================================================<br></div><br><div class="gmail_quote">On Mon, Jul 23, 2012 at 10:08 PM, Samba <span dir="ltr"><<a href="mailto:saasira@gmail.com" target="_blank">saasira@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Lars,<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><br>I noticed that the previous patch that I sent was only syncing the lowest level subdirectory [all files under it] in which one of the files matching the configured pattern have changed. It is not the parent directories even the parent directory is missing on the slave side.<br>
<br>I tried something like:<br><br>in update.c<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote style="margin:0 0 0 40px;border:none;padding:0px">strncpy(temp,filename,filename_length+1);</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px"> 349 do {</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"> 350 get_parent_child_from_path(parent_dirname,NULL,temp);</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px"> 351 csync_debug(0,"missing parent dir : %s; temp %s",parent_dirname, temp);</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
352 strncpy(temp,parent_dirname,strlen(parent_dirname)+1);</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"> 353 conn_printf("SIG %s %s\n", url_encode(key), url_encode(temp));</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px"> 354 last_conn_status=read_conn_status(temp, peername);</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
355 } while( last_conn_status==CONNECTION_RESPONSE_PARENT_DIR_MISSING);</blockquote><div><br></div></blockquote>in daemon.c<br><br><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
if ( lstat_strict(filepath, &st) != 0 ) {</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">555 csync_debug(3,"error : %s while stating %s ",strerror(errno),filepath);</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">556 if ( errno == ENOENT ) {</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">557 struct stat sb;</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">558 char parent_dirname[strlen(filepath)];</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
559 get_parent_child_from_path(parent_dirname,NULL,filepath);</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">560 if ( lstat_strict(prefixsubst(parent_dirname), &sb) != 0 ) {</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">561 conn_printf("OK (parent_dir_missing).\n---\noctet-stream 0 \n");</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
562 } else {</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">563 conn_printf("OK (path_not_found).\n---\noctet-stream 0\n") ;</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">564 }</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">565 } else {</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">566 cmd_error = strerror(errno);</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">567 }</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">568 break;</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">569 } else if ( csync_check_pure(tag[2]) ) {</blockquote>
<blockquote style="margin:0 0 0 40px;border:none;padding:0px">570 conn_printf("OK (path_not_found).\n---\noctet-stream 0\n");</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">
571 break;</blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px">572 }</blockquote><br> so that I can find the last available parent directory on the Slave server and can mark anything under that directory as dirty; but for some reason the slave does not return the message "OK (parent_dir_missing)" for the second "command/request". For the first SIG command it does indeed return "OK (parent_dir_missing)" but after that the response for second SIG is "---\n" which is strange. Is the csync protocol stateful and needs to be communicated in a particular order? If not, then can you please guide me as to what can be done to achieve what I'm expecting? i.e. repeatedly SIG the slave/peer for finding if the parent directory of a given path exists or if it is just the current path that is missing or the path exists but has contents different from that on master. If SIG is not the proper command/signal that needs to be used for this purpose, then do let me know which other command can work here.<br>
<br>I think the only thing that needs to be done is find the last available parent directory from the path of the file to be synced and then mark all the files and subdirectories matching the configured patterns as dirty so that all these files and subdirs gets synced up with the peer in the next round of cron job.<br>
<br><br></blockquote><div class="im HOEnZb">Thanks and Regards,<br>Samba<br><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><br></blockquote>===================================================================================<br>
</div><div class="HOEnZb"><div class="h5"><div class="gmail_quote">
On Thu, Jul 19, 2012 at 12:42 PM, Lars Ellenberg <span dir="ltr"><<a href="mailto:lars.ellenberg@linbit.com" target="_blank">lars.ellenberg@linbit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Tue, Jul 17, 2012 at 10:44:13PM +0530, Samba wrote:<br>
> Hi Lars,<br>
><br>
> I have updated the patch to also sync the complete directory tree, but in<br>
> the subsequent run.<br>
<br>
</div>Thanks.<br>
<br>
Sorry for response times, swamped.<br>
Though I should find some time for csync2 next week.<br>
<div><br>
> I tried to make it sync in the same run by trying to invoke<br>
> csync_update_file_mod funtion but was getting "Format Error", so left it<br>
> with syncing in the subsequent run by marking those children<br>
> [files/folders] as dirty.<br>
><br>
> I also made a minor fix related to creating the configured backup directory<br>
> instead of failing with an unrecognizable error message "Format Error".<br>
><br>
> I'm confident that this patch makes csync2 more stabler and leavese very<br>
> few chances of failing.<br>
><br>
> I hope this patch gets approved and finally makes its way to the trunk soon.<br>
><br>
><br>
> Thanks and Regards,<br>
> Samba<br>
><br>
><br>
> PS:<br>
><br>
> A note about handling deletes with auto-mode younger/older --- perhaps we<br>
> may not have a clean way of identifying whether the deletion on one node is<br>
> younger than a modification on the other node when we are relying on<br>
> scheduled mechanism like cron but we would certainly be able to distinguish<br>
> when we integrate an inode eventing mechanism like lsync for populating the<br>
> "hints" table. I agree that we should keep it for future though.<br>
<br>
</div>In that case you are talking about a tight race.<br>
<br>
If you call csync2 -u "occasionally" only, then conflicts can happen.<br>
<br>
But if you trigger it "immediately" (via inotify),<br>
to have a conflict there,<br>
you'd really need this to happen "simultaneously".<br>
<br>
Because usually, you'd have either the deletion first (and propagated),<br>
and then not a modify, but a (re-)create (propagated).<br>
Or you'd have the modify (also propagated),<br>
and then a delete (propagated).<br>
<br>
So that would mean you really don't want to use csync2 as arbitrator<br>
there, but fix your application to use some means of cooperation<br>
("locking"?) *outside* of csync2.<br>
<div><div><br>
<br>
Lars<br>
_______________________________________________<br>
Csync2 mailing list<br>
<a href="mailto:Csync2@lists.linbit.com" target="_blank">Csync2@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/csync2" target="_blank">http://lists.linbit.com/mailman/listinfo/csync2</a><br>
</div></div></blockquote></div><br>
</div></div></blockquote></div><br>