<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
One "fun-fact": according to iostat disk sdb writes about 15% less
blocks per second than the other two SATAs backing the RAID5. <br>
as the only RAID1-devices on this three disks are /boot and swap
(both not in use at all) I ask myself how that might be possible...
especially as before deleting all the files the three devices had
the same blocks/sec-usage (see attachment).<br>
<br>
Am 2011-01-28 20:44, schrieb Joseph Hauptmann:
<blockquote cite="mid:4D431C87.2020903@digiconcept.net" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
Yes, I did try that. Doesn't make much of a (speed) difference.<br>
<br>
It seems, that the problem is less that rm gets stuck for good,
but that it takes really long breaks (about 20 sec.) while
deleting - during those breaks the whole partition is stuck and
iostat reports 100% utilization compared to ~95% while actually
deleting files. Could the "hang-time" be DRBD writing
meta-information (internal in my case) and blocking every other
access as long the meta-data isn't written to the disk? Of course
there is also the ext3-journal that has to be written, but still I
don't see why it should take that long: I'm currently timing how
long it takes to delete a subdir with 285868 block-sized files in
it (already more than 30 min).<br>
<br>
dmesg is clear, so it does not seem to be a SATA reset.<br>
<br>
any other ideas?<br>
<br>
<br>
<br>
<br>
Am 2011-01-28 20:02, schrieb Moti Levy:
<blockquote
cite="mid:AANLkTi=0+cPf2YMTcKujsPw_vpRaiHvRGEVSSB3K=QwA@mail.gmail.com"
type="cite"><font face="trebuchet ms,sans-serif">Have you tried
: </font>
<div><font face="trebuchet ms,sans-serif">find dirname -type f
-exec rm {} \;</font></div>
<div><font face="trebuchet ms,sans-serif"><br>
</font></div>
<div><br>
<div class="gmail_quote"> On Fri, Jan 28, 2011 at 1:46 PM,
Joseph Hauptmann <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:joseph@digiconcept.net">joseph@digiconcept.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt
0.8ex; border-left: 1px solid rgb(204, 204, 204);
padding-left: 1ex;"> Hello DRBD-users worldwide...<br>
<br>
I've been using DRBD almost a year now, until now without
problems that I couldn't resolve myself.<br>
But now I ran into quite a serious problem and I'm
interested if someone else experienced something similar
with or without DRBD (as of course I can't really be sure
that DRBD is the problem):<br>
<br>
A few months ago a colleague of mine forgot to activate a
cronjob, that deletes a couple thousand very small
temporary files each night on a DRBD-device. Now I have a
directory with, I guess more than a million files, which
wouldn't be so bad, if rm -rf {dir}/ could delete it. But
sadly that is not the case.<br>
rm gets stuck after it deleted a few hundred files and
doesn't resume operation. Furthermore the all IO-access on
the DRBD-device is complete stuck until the rm process is
killed.<br>
<br>
I've already disconnected all resources from it's peer and
shut down most of the non essential services on the
machine.<br>
<br>
It's running Debian Lenny with<br>
<br>
uname -a<br>
Linux <a moz-do-not-send="true" href="http://srv1.xxx.at"
target="_blank">srv1.xxx.at</a> 2.6.26-2-openvz-amd64 #1
SMP Wed May 12 18:14:56 UTC 2010 x86_64 GNU/Linux<br>
<br>
cat /proc/drbd<br>
version: 8.3.7 (api:88/proto:86-91)<br>
GIT-hash: ea9e28dbff98e331a62bcbcc63a6135808fe2917 build
by <a moz-do-not-send="true"
href="mailto:root@srv1.xxx.at" target="_blank">root@srv1.xxx.at</a>,
2010-03-28 21:47:13<br>
0: cs:WFConnection ro:Primary/Unknown
ds:UpToDate/DUnknown C r----<br>
ns:1875795496 nr:0 dw:225995436 dr:566154981
al:105639961 bm:11019801 lo:2 pe:0 ua:0 ap:1 ep:1 wo:b
oos:1242040<br>
1: cs:StandAlone ro:Secondary/Unknown
ds:UpToDate/DUnknown r----<br>
ns:0 nr:31796784 dw:31796784 dr:2253416 al:0 bm:1134
lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0<br>
2: cs:StandAlone ro:Secondary/Unknown
ds:UpToDate/DUnknown r----<br>
ns:0 nr:57709884 dw:143774088 dr:8480 al:0 bm:50 lo:0
pe:0 ua:0 ap:0 ep:1 wo:d oos:0<br>
<br>
The filesystem on resource 0 is ext3 with a block size of
4096 and lies on a SW-RAID5 (far from ideal - I know).<br>
<br>
<br>
Atm. I'm using a bash-hack, that kills the rm-process
every 30 seconds and restarts it as long as the directory
still exists.<br>
<br>
Thanks for any hints to what might cause this problem.<br>
<br>
Joe<br>
<br>
-- <br>
Joseph Hauptmann<br>
<br>
/digiconcept/ - GmbH.<br>
1080 Wien<br>
Blindengasse 52/1<br>
<br>
Tel. +43 1 218 0 212 - 24<br>
Fax +43 1 218 0 212 - 10<br>
<br>
_______________________________________________<br>
drbd-user mailing list<br>
<a moz-do-not-send="true"
href="mailto:drbd-user@lists.linbit.com" target="_blank">drbd-user@lists.linbit.com</a><br>
<a moz-do-not-send="true"
href="http://lists.linbit.com/mailman/listinfo/drbd-user"
target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Joseph Hauptmann
/digiconcept/ - GmbH.
1080 Wien
Blindengasse 52/1
Tel. +43 1 218 0 212 - 24
Fax +43 1 218 0 212 - 10 </pre>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
drbd-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a>
<a class="moz-txt-link-freetext" href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Joseph Hauptmann
/digiconcept/ - GmbH.
1080 Wien
Blindengasse 52/1
Tel. +43 1 218 0 212 - 24
Fax +43 1 218 0 212 - 10 </pre>
</body>
</html>