Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
I'm having a bit of a problem with my sync rate. It's never been
especially fast, but its even slower than usual. Slower than any of the
previous posts in the archive, so I felt it would be okay to post my
version of a common issue. (I hope I'm right.)
This morning watchdog caused my primary to fail, and the secondary started
to take over. Before it could complete, the primary came back up, so I
have one node on the backup as primary, and the other 9 are on the main
server as primary. For about 18 hours now it's been resyncing, and there's
still a lot of time left. I understand that splitting the bandwidth among
all of these nodes will slow things down a bit, but I've only occasionally
seen the transer rate hit 124 K/sec.
----------------------
My /proc/drbd currently looks like this:
0: cs:SyncingAll st:Primary/Secondary ns:1126468 nr:0 dw:0 dr:1126500 pe:75
ua:0
[=========>..........] sync'ed: 47.9% (1203/2304)M
finish: 14:14:26h speed: 23 (17) K/sec
1: cs:SyncingAll st:Primary/Secondary ns:1135108 nr:0 dw:0 dr:1135140
pe:123 ua:0
[==========>.........] sync'ed: 54.2% (939/2048)M
finish: 10:19:23h speed: 25 (17) K/sec
2: cs:SyncingAll st:Primary/Secondary ns:1122156 nr:0 dw:0 dr:1122180
pe:146 ua:0
[=====>..............] sync'ed: 26.8% (3000/4096)M
finish: 37:25:57h speed: 21 (17) K/sec
3: cs:Connected st:Primary/Secondary ns:1048576 nr:0 dw:0 dr:1048576 pe:0
ua:0
4: cs:Connected st:Primary/Secondary ns:1048576 nr:0 dw:0 dr:1048576 pe:0
ua:0
5: cs:SyncingAll st:Primary/Secondary ns:1117568 nr:0 dw:0 dr:1117620
pe:104 ua:0
[=>..................] sync'ed: 6.7% (15292/16384)M
finish: 195:45:38h speed: 21 (16) K/sec
6: cs:SyncingAll st:Primary/Secondary ns:1137504 nr:0 dw:0 dr:1137540
pe:152 ua:0
[==>.................] sync'ed: 13.6% (7081/8192)M
finish: 75:56:07h speed: 24 (17) K/sec
7: cs:Connected st:Primary/Secondary ns:524288 nr:0 dw:0 dr:524288 pe:0
ua:0
8: cs:SyncingAll st:Secondary/Primary ns:0 nr:281672 dw:281672 dr:0 pe:0
ua:0
[=>..................] sync'ed: 6.8% (3821/4096)M
finish: 65:14:01h speed: 9 (4) K/sec
9: cs:SyncingAll st:Primary/Secondary ns:1190208 nr:0 dw:0 dr:1190220 pe:86
ua:0
[==>.................] sync'ed: 14.3% (7029/8192)M
finish: 81:25:34h speed: 23 (18) K/sec
10: cs:SyncingAll st:Primary/Secondary ns:1141812 nr:0 dw:0 dr:1141860
pe:51 ua:0
[=>..................] sync'ed: 6.9% (15268/16384)M
finish: 163:43:10h speed: 24 (17) K/sec
11: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0
12: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0
13: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0
14: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0
15: cs:Unconfigured st:Secondary/Unknown ns:0 nr:0 dw:0 dr:0 pe:0 ua:0
----------------------
Meanwhile my web server is down, so to say there's some urgency to my
getting this resolved is an understatement.
I'd really appreciate any comments or advice. I've tried drbdsetup and
changing the min, max, and nice sizes, but it doesn't affect anything.
Hopefully once this is figured out I can see where my data went. I was
expecting it to be present when the nodes stopped resyncing (otherwise
what's the point?), but it looks like they'll have to be recovered from a
backup. Thanks very much....
Jesse
----