Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
My thoughts are if this is hardware related it is probably network link
or memory.
First network link:
To check out the crossover links I ran some bing tests:
On the secondary machine:
[root at grp-01-30-01 tmp]# bing 192.168.255.1 192.168.255.2
BING 192.168.255.1 (192.168.255.1) and 192.168.255.2 (192.168.255.2)
44 and 108 data bytes
1024 bits in 0.000ms
1024 bits in 0.006ms: 170666667bps, 0.000006ms per bit
1024 bits in 0.008ms: 128000000bps, 0.000008ms per bit
1024 bits in 0.010ms: 102400000bps, 0.000010ms per bit
1024 bits in 0.013ms: 78769231bps, 0.000013ms per bit
1024 bits in 0.012ms: 85333333bps, 0.000012ms per bit
1024 bits in 0.011ms: 93090909bps, 0.000011ms per bit
1024 bits in 0.012ms: 85333333bps, 0.000012ms per bit
1024 bits in 0.011ms: 93090909bps, 0.000011ms per bit
1024 bits in 0.012ms: 85333333bps, 0.000012ms per bit
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
1024 bits in 0.011ms: 93090909bps, 0.000011ms per bit
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
1024 bits in 0.013ms: 78769231bps, 0.000013ms per bit
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
--- 192.168.255.1 statistics ---
bytes out in dup loss rtt (ms): min avg max
44126741126741 0% 0.055 0.057 0.642
108126741126741 0% 0.053 0.056 0.543
--- 192.168.255.2 statistics ---
bytes out in dup loss rtt (ms): min avg max
44126741126740 0% 0.086 0.098 6.134
108126740126740 0% 0.099 0.108 4.100
--- estimated link characteristics ---
warning: rtt big host1 0.053ms < rtt small host2 0.055ms
estimated throughput 78769231bps
minimum delay per packet 0.022ms (1738 bits)
average statistics (experimental) :
packet loss: small 0%, big 0%, total 0%
warning: rtt big host1 0.056ms < rtt small host2 0.057ms
average throughput 113777778bps
average delay per packet 0.032ms (2531 bits)
weighted average throughput 113777329bps
and again on the secondary machine:
--- 192.168.255.1 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 52144 52144 0% 0.055 0.058 0.931
108 52144 52144 0% 0.054 0.056 0.550
--- 192.168.255.2 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 52144 52144 0% 0.087 0.099 6.150
108 52144 52143 0% 0.099 0.108 5.049
--- estimated link characteristics ---
warning: rtt big host1 0.054ms < rtt small host2 0.055ms
estimated throughput 85333333bps
minimum delay per packet 0.024ms (2027 bits)
average statistics (experimental) :
packet loss: small 0%, big 0%, total 0%
warning: rtt big host1 0.056ms < rtt small host2 0.058ms
average throughput 113777778bps
average delay per packet 0.033ms (2822 bits)
weighted average throughput 113776687bps
========================================================
And now from the primary machine:
[root at grp-01-30-02 tmp]# bing 192.168.255.2 192.168.255.1
BING 192.168.255.2 (192.168.255.2) and 192.168.255.1 (192.168.255.1)
44 and 108 data bytes
1024 bits in 0.000ms
1024 bits in 0.002ms: 512000000bps, 0.000002ms per bit
1024 bits in 0.001ms: 1024000000bps, 0.000001ms per bit
1024 bits in 0.011ms: 93090909bps, 0.000011ms per bit
1024 bits in 0.010ms: 102400000bps, 0.000010ms per bit
1024 bits in 0.009ms: 113777778bps, 0.000009ms per bit
1024 bits in 0.010ms: 102400000bps, 0.000010ms per bit
1024 bits in 0.009ms: 113777778bps, 0.000009ms per bit
1024 bits in 0.010ms: 102400000bps, 0.000010ms per bit
1024 bits in 0.009ms: 113777778bps, 0.000009ms per bit
1024 bits in 0.010ms: 102400000bps, 0.000010ms per bit
1024 bits in 0.011ms: 93090909bps, 0.000011ms per bit
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
1024 bits in 0.013ms: 78769231bps, 0.000013ms per bit
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
bing: packet (139 bytes) from unexpected host 192.168.1.1
--- 192.168.255.2 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 42019 42019 0% 0.055 0.058 1.622
108 42019 42019 0% 0.053 0.056 0.719
--- 192.168.255.1 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 42019 42019 0% 0.086 0.100 11.247
108 42019 42018 0% 0.099 0.112 55.077
--- estimated link characteristics ---
warning: rtt big host1 0.053ms < rtt small host2 0.055ms
estimated throughput 78769231bps
minimum delay per packet 0.022ms (1738 bits)
average statistics (experimental) :
packet loss: small 0%, big 0%, total 0%
warning: rtt big host1 0.056ms < rtt small host2 0.058ms
average throughput 85333333bps
average delay per packet 0.033ms (2626 bits)
weighted average throughput 85332318bps
[root at grp-01-30-02 tmp]#
and again on the primary machine:
--- 192.168.255.2 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 52317 52317 0% 0.054 0.057 1.289
108 52317 52317 0% 0.053 0.056 1.363
--- 192.168.255.1 statistics ---
bytes out in dup loss rtt (ms): min avg max
44 52317 52317 0% 0.086 0.100 10.318
108 52317 52316 0% 0.099 0.110 13.140
--- estimated link characteristics ---
warning: rtt big host1 0.053ms < rtt small host2 0.054ms
estimated throughput 78769231bps
minimum delay per packet 0.023ms (1817 bits)
average statistics (experimental) :
packet loss: small 0%, big 0%, total 0%
warning: rtt big host1 0.056ms < rtt small host2 0.057ms
average throughput 102400000bps
average delay per packet 0.034ms (2647 bits)
weighted average throughput 102399021bps
From these results I would think that the network link is ok.
==================================================
Next memory: I'm going to run memtest86 on both machines.
grp-01-30-01: failed test #5 on 3 addresses, passed all other memory
tests. I've read that test #5 (block move, 64 moves) is not that critical.
grp-01-30-02: also failed test #5 on some addresses, passed all other
memory tests.
So how critical is test #5 to drbd?