[Csync2] speed of "-cr /"

[Lars Ellenberg]

On Thu, Jun 10, 2010 at 11:46:43PM +1000, Aristedes Maniatis wrote:
> On 10/06/10 7:53 PM, Lars Ellenberg wrote:
> >On Thu, Jun 10, 2010 at 06:29:08PM +1000, Aristedes Maniatis wrote:
> >>>  On 10/06/10 6:02 PM, Lars Ellenberg wrote:
> >>>>  >For -R, its just config parsing, and SQL.
> >>>>  >
> >>>>  >For -c, it is one lstat for every record fetched.
> >>>>  >100000 lstat calls may take some time;-)
> >>>
> >>>  In that case we've come a bit full circle with this conversation. Where we started:
> >>>
> >>>  * why does csync2 ignore the 'include' parameters when running -c?
> >why do you have 100000 "stale" no longer relevant records in your database?
> 
> They aren't stale, they are in a different group. Isn't that the point
> of having groups? So you can run one part of the configuration without
> having to run the whole thing?


First time you mention groups.

If you had been less assumptious, less accusing, less ranting about what
does not work and how stupid the rest of the world is,
but more precise about what you are doing, and included your conf file
right away (instead of just one single include line in some followup),
you could have been here a few days ago ;-)

Csync2 does two scans:
 it checks for deletions, basically select filenam from file,
   is filename still present?  no-> mark as dirty.
 it checks for modifications (including additions), basically
   scanning directories, comparing with database entries.

If you don't explicitly state "-G ", csync2 defaults to all groups.

But even if you do "-G single_entry_group", that currently optimizes
only the "check for modifications/additions".

The "check for deletions" currently does not care for groups.

If different configs had been supported from the very start,
groups probably had not been invented in the first place.

> I'd rather not create a dozen config files with a dozen databases.

Why not?

Converting your truely independent groups with many files each
into their own configs instead seems to be a good idea,
even just from a sqlite maintenance, performance, and congestion
point of view.

I don't see why it would pose an administration overhead, either.

-- 
: Lars Ellenberg
: LINBIT | Your Way to High Availability
: DRBD/HA support and consulting http://www.linbit.com

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.