[DRBD-user] 3 node stacked setup and dual primary issues

[Cristian Mammoli]

Hi, as far as I understand a 3 node steup with dual primary and 3rd node 
for disaster recovery is not possible:

http://www.gossamer-threads.com/lists/drbd/users/22188#22188

So I tried the following setup as suggested by Lars:

drbd-1: 192.168.3.111 (first node)
drbd-2: 192.168.3.112 (second node)
drbd-dr: 192.168.3.113 (disaster recovery node)

drbd-1 and drbd-2 should be Primary/Primary (I need it as a backing 
store for kvm VMs)

drbd-2 replicates with drbd-dr in Primary/Secondary mode and protocol A
drbd-2 replicates resource stacked on the previous one with drbd-1 and 
protocol C


Scheme:

drbd1 <---C--->	drbd1

		drbd0 ---A---->	drbd0	
		
node-drbd-1	node-drbd-2	node-drbd-dr

The resources start up and sync. The i tried the following:

format drbd1 device on drbd-1
mount drbd1 device on drbd-1
create test file drbd-1-test
umount drbd1 device on drbd-1

mount drbd1 device on drbd-2
create test file drbd-2-test
I can see both drbd-1-test and drbd-2-test
umount drbd1 device on drbd-2

demote and stop stacked drbd1 resource on drbd-1 and drbd-2
promote drbd0 resource on drbd-dr
mount drbd0 device on drbd-dr
create test file drbd-dr-test
I can see drbd-1-test drbd-2-test drbd-dr-test

demote drbd0 resource on drbd-dr
start and promote drbd1 resource on drbd-1 and drbd-2

mount drbd1 on drbd-2
I can see drbd-1-test drbd-2-test drbd-dr-test
umount drbd1 on drbd-2

mount drbd1 on drbd-1
I can see drbd-1-test drbd-2-test ONLY!!

If I invalidate the resource on drbd-1 and do a full resync the 3rd file 
appear.

Config follows:
resource drbd-dr {
   protocol C;

   startup {
     become-primary-on       drbd-2.apra.it;
   }

   on drbd-2.apra.it {
     device     /dev/drbd0;
     disk       /dev/vdb;
     address    192.168.3.112:7788;
     meta-disk internal;
   }

   on drbd-dr.apra.it {
     device    /dev/drbd0;
     disk      /dev/vdb;
     address   192.168.3.113:7788;
     meta-disk internal;
   }
}

resource drbd {
   startup {
     become-primary-on       both;
   }
   net {
     allow-two-primaries;
   }

   protocol C;

   stacked-on-top-of drbd-dr {
     device     /dev/drbd1;
     address    192.168.3.112:7789;
   }

   on drbd-1.apra.it {
     device     /dev/drbd1;
     disk       /dev/vdb;
     address    192.168.3.111:7789; # Public IP of the backup node
     meta-disk  internal;
   }
}




-- 
Cristian Mammoli
APRA SISTEMI srl
Via Brodolini,6 Jesi (AN)
tel dir. +390731719822

Web www.apra.it
e-mail c.mammoli at apra.it