[Csync2] I/O Error 'Too many open files'

[Clifford Wolf]

Hi,

On Fri, Dec 30, 2005 at 02:41:08PM +0100, Yann Sarazin wrote:
> I'm trying to use csync2 to synchronize about 1,300,000 files but I've 
> got the following error :
> 
> <24924> ERROR: tmpfile() didn't return a valid file handle!
> <24924> I/O Error 'Too many open files' while creating new data temp 
> file in rsync-patch: /home/www/correzrt/correzrt_price_list_4734.pdf
> <24924> ERROR: tmpfile() didn't return a valid file handle!
> [...]

I've now done some profiling in all three csync2 operation modes (check,
update, daemon) and it does not look like csync2 is leaking file
descriptors (max. 10 open file descriptors at a time).

So that looks like the system-wide limit for open files is reached on your
box independent of csync2. But csync2 had the bad luck to request a new fd
when this limit was reached.

Find the program which does leak file descriptors or (if you really need
that much open files) incrase the system wide limit.

> This error occur after a few minutes of successful synchronization, and 
> after the error occurred the csync2 -ii process continue running 
> indefinitely using 100% of the CPU.

that's the usual behavior of librsync when it can not do i/o.

I've now changed the error handling in that code path so we avoid calling
any librsync functions when tmpfile() fails.

yours,
 - clifford

-- 
  _________  ____ __      __  $_ = q 7nz!y="Ccv'Dpgiutvcbb'oj'Pocj'*'qqq[
 / ___/ __ \/ __ `/ | /| / /  ej`q[iv`";!|=1;!y=~t%].[%nz!z=!1;gps]1..6[{
/ /__/ / / / /_/ /| |/ |/ /   qsjou"!z\c";tfmfdu]voefg,voefg,voefg,0.05[;
\___/_/ /_/\__, / |__/|__/    !z=~t/].[/dis]pse]!1[^!_[/fh;}qsjou !z;%fh;
 CCC Wien /____/www.cngw.org  qsjou "\o"7;s/\n//g;y/[b-za]!/)a-z($/;eval;
 
Programming is like sex: One mistake and you have to support it your lifetime.