[Csync2] Adding a huge amount of data to an existing csync2/lsyncd setup

[Kevin Cackler]

Thanks, Jerry. I do have the paper and actually did use that section of 
it to initialize these new files, but csync2 still wanted to sync 
everything up on the next sync. I figured out late last night that the 
file creation times were different on each of the hosts, and because of 
that, csync2 had to sync each file to get everything caught up. I won't 
make that mistake next time I have to do this :)

Thanks for your help.

Jerry Chin wrote:
>
> Hey! Do you have the csync2 paper at hand which came along with the
> package (or check out this link
> http://oss.linbit.com/csync2/paper.pdf) ? thoroughly read chapter *5.5
> Bootstrapping large setups* will extremely helpful.
>
> The chapter will show you how to add new files to status db without
> marking them as dirty, so they will not be synchronized in the next loop.
>
> Feel free to contact me, if you have more questions.
>
>
>     /*Regards,
>     Jerry Chin.
>
>     */_
>     ​
>     下述
>     ​​
>     真理
>     ​​
>     不证自明：凡
>     ​为​
>     人
>     ​类，​
>     生而平等，秉造物者之赐，拥
>     ​​
>     诸
>     ​不
>     可
>     ​剥夺​
>     之权利，包含生命权、自由权、
>     ​及
>     追
>     ​求
>     幸福权
>     ​。​
>     ​​
>     _/
>     /
>
>
> On Wed, May 3, 2017 at 4:45 AM, Kevin Cackler <kevin at techdaddies.com
> <mailto:kevin at techdaddies.com>> wrote:
>
>     What is the best method for adding a lot of data to an already
>     existing setup? I'm talking about 50GB here and probably around
>     60,000 files. If I just add the files to a node in the cluster, it
>     takes many hours for the data to sync to the rest of the nodes. I
>     would be fine with stopping lsyncd on all of the nodes and then
>     copying the data manually to each node, but I'm unsure of how to
>     then make csync2 aware of the new files (And aware that the files
>     are identical and OK for syncing).
>     _______________________________________________
>     Csync2 mailing list
>     Csync2 at lists.linbit.com <mailto:Csync2 at lists.linbit.com>
>     http://lists.linbit.com/mailman/listinfo/csync2
> <http://lists.linbit.com/mailman/listinfo/csync2>
>


> Jerry Chin <mailto:metrxqin at gmail.com>
> May 2, 2017 at 8:22 PM
> Hey! Do you have the csync2 paper at hand which came along with the 
> package (or check out this link 
> http://oss.linbit.com/csync2/paper.pdf) ? thoroughly read chapter *5.5 
> Bootstrapping large setups* will extremely helpful.
>
> The chapter will show you how to add new files to status db without 
> marking them as dirty, so they will not be synchronized in the next loop.
>
> Feel free to contact me, if you have more questions.
>
>
>
>
> Kevin Cackler <mailto:kevin at techdaddies.com>
> May 2, 2017 at 3:45 PM
> What is the best method for adding a lot of data to an already 
> existing setup? I'm talking about 50GB here and probably around 60,000 
> files. If I just add the files to a node in the cluster, it takes many 
> hours for the data to sync to the rest of the nodes. I would be fine 
> with stopping lsyncd on all of the nodes and then copying the data 
> manually to each node, but I'm unsure of how to then make csync2 aware 
> of the new files (And aware that the files are identical and OK for 
> syncing).
> _______________________________________________
> Csync2 mailing list
> Csync2 at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/csync2

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/csync2/attachments/20170503/1160ec14/attachment.htm>