[DRBD-user] Xen migration with drbd

Wouter de Geus wouter-linbit.com at owl-ict.nl
Tue Apr 29 16:22:36 CEST 2014

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


* Lars Ellenberg <lars.ellenberg at linbit.com> [2014-04-29 16:03:47 +0200]:

> > My problem: migrating a host to the other machine.
> > It works, but somehow it fails to demote the old host to secondary status, so after the migration has completed (this part works) I have a situation
> > with 2 hosts in primary mode.

Woops, I remembered wrong. What goes wrong is exactly the opposite:
Both members end up in secondary mode somehow.
Sorry for that!

> You should have some logs along with this.
> Maybe the script correctly "asked" DRBD to become secondary,
> but DRBD refused, as it still was "in-use" by "someone".

I don't have much logs, what I have:

Sending host:
---
# cat xl-green.log
Waiting for domain green (domid 10) to die [pid 25883]
Domain 10 has shut down, reason code 2 0x2
Domain has suspended.
Done. Exiting now
---

Receiving host:
---
# cat xl-green--incoming.log
Waiting for domain green--incoming (domid 2) to die [pid 8490]
---

Syslog of sending host:
---
Apr 29 16:14:24 iejoor kernel: [5249166.103298] xenbr0: port 2(green0.0) entered disabled state
Apr 29 16:14:24 iejoor kernel: [5249166.116765] xenbr0: port 2(green0.0) entered disabled state
Apr 29 16:14:24 iejoor kernel: [5249166.116826] device green0.0 left promiscuous mode
Apr 29 16:14:24 iejoor kernel: [5249166.116832] xenbr0: port 2(green0.0) entered disabled state
Apr 29 16:14:24 iejoor kernel: [5249166.184401] xenbr1: port 2(green1.1) entered disabled state
Apr 29 16:14:24 iejoor kernel: [5249166.184902] device green1.1 left promiscuous mode
Apr 29 16:14:24 iejoor kernel: [5249166.184930] xenbr1: port 2(green1.1) entered disabled state
Apr 29 16:14:25 iejoor kernel: [5249166.763305] block drbd1: role( Primary -> Secondary )
Apr 29 16:14:25 iejoor kernel: [5249166.763381] block drbd1: bitmap WRITE of 0 pages took 0 jiffies
Apr 29 16:14:25 iejoor kernel: [5249166.796550] block drbd1: 0 KB (0 bits) marked out-of-sync by on disk bit-map.
Apr 29 16:14:26 iejoor ntpd[800]: Deleting interface #23 green0.0, fe80::fcff:ffff:feff:ffff#123, interface stats: received=0, sent=0, dropped=0, active_time=348556 secs
Apr 29 16:14:26 iejoor ntpd[800]: Deleting interface #22 green1.1, fe80::fcff:ffff:feff:ffff#123, interface stats: received=0, sent=0, dropped=0, active_time=348556 secs
---

Syslog of receiving host:
---
Apr 29 16:14:24 winnie kernel: [3350190.345466] device green0.0 entered promiscuous mode
Apr 29 16:14:24 winnie kernel: [3350190.346873] IPv6: ADDRCONF(NETDEV_UP): green0.0: link is not ready
Apr 29 16:14:24 winnie kernel: [3350190.352170] device green1.1 entered promiscuous mode
Apr 29 16:14:24 winnie kernel: [3350190.353647] IPv6: ADDRCONF(NETDEV_UP): green1.1: link is not ready
Apr 29 16:14:24 winnie kernel: [3350190.423967] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi) persistent grants
Apr 29 16:14:24 winnie sshd[8211]: Received disconnect from 192.168.2.2: 11: disconnected by user
Apr 29 16:14:24 winnie kernel: [3350190.514697] IPv6: ADDRCONF(NETDEV_CHANGE): green0.0: link becomes ready
Apr 29 16:14:24 winnie kernel: [3350190.514784] xenbr0: port 2(green0.0) entered forwarding state
Apr 29 16:14:24 winnie kernel: [3350190.514798] xenbr0: port 2(green0.0) entered forwarding state
Apr 29 16:14:24 winnie kernel: [3350190.569476] IPv6: ADDRCONF(NETDEV_CHANGE): green1.1: link becomes ready
Apr 29 16:14:24 winnie kernel: [3350190.569538] xenbr1: port 2(green1.1) entered forwarding state
Apr 29 16:14:24 winnie kernel: [3350190.569551] xenbr1: port 2(green1.1) entered forwarding state
Apr 29 16:14:25 winnie kernel: [3350191.209910] block drbd1: peer( Primary -> Secondary ) 
Apr 29 16:14:27 winnie ntpd[709]: Listen normally on 12 green1.1 fe80::fcff:ffff:feff:ffff UDP 123
Apr 29 16:14:27 winnie ntpd[709]: Listen normally on 13 green0.0 fe80::fcff:ffff:feff:ffff UDP 123
Apr 29 16:14:27 winnie ntpd[709]: peers refreshed
Apr 29 16:14:39 winnie kernel: [3350205.527372] xenbr0: port 2(green0.0) entered forwarding state
Apr 29 16:14:39 winnie kernel: [3350205.591358] xenbr1: port 2(green1.1) entered forwarding state
---

> That someone frequently turns out to be some udev triggered device scan.
> 
> Please find the logs,
> or add your own "echo >> some-log" style debugging to the script,
> then see why it does not work as expected.

I'll try to see if I can figure out some more details.

> Maybe we simply need to add some sleep (or udevadm settle with timeout)
> and retry to the drbdadm secondary.

My apologies, that was not the issue :)
But maybe it's something similar.

Regards,

Wouter.



More information about the drbd-user mailing list