Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Maurice Volaski wrote:
> Can you please post the configuration you use for the other daemons do
> this to the drbd mailing list? I want to do it too.
Hello,
Today i tried to use DRBD on loopback interfaces with OSPF routing.
It turns out, DRBD is not yet ready for such a setup (see below).
First of all, please really think about wether you need and want this
added complexity of ospf. It is a tradeoff: You add ospf as a new
possible error source. You get some extra protection against network
failures.
Here is a somewhat verbose procedure (for Redhat/Whitebox):
- Install quagga
download stable quagga from http://www.quagga.net/ (version 0.98.4 at
time of this writing)
tar -xzvf quagga-*.tar.gz
cd quagga-*
./configure --enable-multipath=4
make
make check
make install
ldconfig -v
useradd quagga
cp redhat/ospfd.init /etc/rc.d/init.d
cp redhat/zebra.init /etc/rc.d/init.d
chmod 755 /etc/rc.d/init.d/*init
vi /etc/sysconfig/quagga
ZEBRA_OPTS="-f /etc/quagga/zebra.conf"
OSPFD_OPTS="-f /etc/quagga/ospfd.conf"
In /etc/rc.d/init.d/zebra.init
change /usr/sbin/zebra to /usr/local/sbin/zebra
in /etc/rc.d/init.d/ospfd.init
change /usr/sbin/ospfd to /usr/local/sbin/ospfd
This command is somewhat Quick and Dirty...
chmod 777 /var/run
mkdir /etc/quagga
cp /usr/local/etc/zebra.conf.sample /etc/quagga/zebra.conf
cp /usr/local/etc/ospfd.conf.sample /etc/quagga/ospfd.conf
(or use the attached example .conf)
chown -R quagga:quagga /etc/quagga
Testrun zebra (should give no errors):
zebra -l
(ctrl-C)
If errors show up, debug with "strace zebra -l",
also look into /var/log/messages
start zebra:
/etc/rc.d/init.d/zebra.init start
chkconfig --add zebra.init
chkconfig zebra.init on
ps -ef | grep zebra
should show a running zebra process
- Configure network interfaces:
Configure your distribution to do the equivalent of the following
ifconfig commands
(e.g. on Redhat/Whitebox create files
/etc/sysconfig/network-scripts/ifcfg-...)
- setup secondary loopback interfaces
Node A:
ifconfig lo:2 192.168.5.1 netmask 255.255.255.255 mtu 1500 up
Node B:
ifconfig lo:2 192.168.5.2 netmask 255.255.255.255 mtu 1500 up
- setup real interfaces
Node A:
ifconfig eth0 192.168.50.1 netmask 255.255.255.0 up
ifconfig eth1 192.168.60.1 netmask 255.255.255.0 up
Node B:
ifconfig eth0 192.168.50.2 netmask 255.255.255.0 up
ifconfig eth1 192.168.60.2 netmask 255.255.255.0 up
- check (NodeA shown):
/sbin/ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
inet 192.168.5.1/32 brd 192.168.5.1 scope global lo:2
4: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:2a:36:d5 brd ff:ff:ff:ff:ff:ff
inet 192.168.50.1/24 brd 192.168.50.255 scope global eth0
5: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:2a:36:cb brd ff:ff:ff:ff:ff:ff
inet 192.168.60.1/24 brd 192.168.60.255 scope global eth1
- Both nodes: Configure zebra (much like Cisco IOS... use ? for help)
telnet localhost 2601
(password = zebra)
ena
(password = zebra)
conf term
! on NodeA:
hostname NodeA
! on NodeB:
hostname NodeB
log syslog
ip forwarding
exit
show running
write mem
exit
- Both Nodes: Start OSPFD
/etc/rc.d/init.d/ospfd.init start
chkconfig --add ospfd.init
chkconfig ospfd.init on
- Both Nodes: Configure OSPFD (simple config)
telnet localhost 2604
(password = zebra)
ena
(password = zebra)
conf term
hostname NodeA
log syslog
router ospf
! Important: the network statements ip and mask MUST MATCH the
! local Nodes interface definitions!
network 192.168.5.1/32 area 0.0.0.0
network 192.168.5.2/32 area 0.0.0.0
network 192.168.50.0/24 area 0.0.0.0
network 192.168.60.0/24 area 0.0.0.0
(ctrl-Z)
show running
write mem
exit
- Connect the networks (if not already done)
- wait 1 min
- check everything:
On NodeA:
telnet localhost 2604
ena
sho ip ospf nei
Neighbor ID Pri State Dead Time Address
Interface RXmtL RqstL DBsmL
192.168.5.2 1 Full/Backup 00:00:39 192.168.50.2
eth0:192.168.50.1 0 0 0
192.168.5.2 1 Full/Backup 00:00:39 192.168.60.2
eth1:192.168.60.1 0 0 0
sho ip ospf route
============ OSPF network routing table ============
N 192.168.5.1/32 [10] area: 0.0.0.0
directly attached to lo
N 192.168.5.2/32 [20] area: 0.0.0.0
via 192.168.50.2, eth0
via 192.168.60.2, eth1
N 192.168.50.0/24 [10] area: 0.0.0.0
directly attached to eth0
N 192.168.60.0/24 [10] area: 0.0.0.0
directly attached to eth1
exit
telnet localhost 2601
ena
sho ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route
C>* 127.0.0.0/8 is directly connected, lo
K>* 169.254.0.0/16 is directly connected, lo
O 192.168.5.1/32 [110/10] is directly connected, lo, 00:16:49
C>* 192.168.5.1/32 is directly connected, lo
O>* 192.168.5.2/32 [110/20] via 192.168.50.2, eth0, 00:08:10
via 192.168.60.2, eth1, 00:08:10
O 192.168.50.0/24 [110/10] is directly connected, eth0, 00:16:49
C>* 192.168.50.0/24 is directly connected, eth0
O 192.168.60.0/24 [110/10] is directly connected, eth1, 00:08:10
C>* 192.168.60.0/24 is directly connected, eth1
exit
On Linux shell again (Note the "proto zebra"):
ip route
192.168.5.2 via 192.168.50.2 dev eth0 proto zebra metric 20 equalize
192.168.50.0/24 dev eth0 proto kernel scope link src 192.168.50.1
192.168.60.0/24 dev eth1 proto kernel scope link src 192.168.60.1
169.254.0.0/16 dev lo scope link
Check on NodeB:
Linux:
ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
inet 192.168.5.2/32 scope global lo:2
4: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:aa:b6:50 brd ff:ff:ff:ff:ff:ff
inet 192.168.50.2/24 brd 192.168.50.255 scope global eth0
5: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:aa:b6:46 brd ff:ff:ff:ff:ff:ff
inet 192.168.60.2/24 brd 192.168.60.255 scope global eth1
Ospf:
telnet localhost 2604
ena
sho ip ospf nei
Neighbor ID Pri State Dead Time Address Interface
RXmtL RqstL DBsmL
192.168.5.1 1 Full/DR 00:00:37 192.168.50.1
eth0:192.168.50.2 0 0 0
192.168.5.1 1 Full/DR 00:00:37 192.168.60.1
eth1:192.168.60.2 0 0 0
sho ip ospf route
============ OSPF network routing table ============
N 192.168.5.1/32 [20] area: 0.0.0.0
via 192.168.50.1, eth0
via 192.168.60.1, eth1
N 192.168.5.2/32 [10] area: 0.0.0.0
directly attached to lo
N 192.168.50.0/24 [10] area: 0.0.0.0
directly attached to eth0
N 192.168.60.0/24 [10] area: 0.0.0.0
directly attached to eth1
exit
Zebra:
telnet localhost 2601
ena
sho ip route
Codes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF,
I - ISIS, B - BGP, > - selected route, * - FIB route
C>* 127.0.0.0/8 is directly connected, lo
O>* 192.168.5.1/32 [110/20] via 192.168.50.1, eth0, 00:04:47
via 192.168.60.1, eth1, 00:04:47
O 192.168.5.2/32 [110/10] is directly connected, lo, 00:04:52
C>* 192.168.5.2/32 is directly connected, lo
O 192.168.50.0/24 [110/10] is directly connected, eth0, 00:04:47
C>* 192.168.50.0/24 is directly connected, eth0
O 192.168.60.0/24 [110/10] is directly connected, eth1, 00:04:47
C>* 192.168.60.0/24 is directly connected, eth1
exit
Linux:
ip route
192.168.5.1 via 192.168.50.1 dev eth0 proto zebra metric 20 equalize
192.168.50.0/24 dev eth0 proto kernel scope link src 192.168.50.2
192.168.60.0/24 dev eth1 proto kernel scope link src 192.168.60.2
- Test wether you have it really,really right:
hostname
NodeA
ssh 192.168.5.2
root at 192.168.5.2's password:
hostname
NodeB
exit
hostname
NodeA
- Setup DRBD
On Both Nodes:
Create /etc/drbd.conf
mknod /dev/drbd0 b 147 0
drbdadm up all
On NodeA:
drbdadm -- --do-what-I-say primary all
netstat -an | grep 7788
tcp 0 0 192.168.5.1:7788 192.168.50.2:32775
ESTABLISHED
tcp 0 0 192.168.50.1:32780 192.168.5.2:7788
ESTABLISHED
This shows a big problem! SRC-IP should be 192.168.5.1 or 192.168.5.2.
The 192.168.50.x IPs mean, network link failover will not work.
AFAIK this can be fixed:
An extra bind-call is needed before the connect-call on the client side
of the tcp connection.
However i am a poor C programmer. Can somebody fix it in drbd?
Best Regards
Matthias
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ospfd.conf.secure
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20050707/cb5b08e0/attachment.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ospfd.conf.simple
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20050707/cb5b08e0/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: zebra.conf
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20050707/cb5b08e0/attachment-0001.asc>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: drbd.conf
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20050707/cb5b08e0/attachment-0001.txt>