[DRBD-user] Cannot synchronize stacked device to backup server with DRBD9

[Artur Kaszuba]

Hi

Im trying to create stacked device with drbd9, but im afraid it could be 
not possible with current drbd-utils or drbd module, IMO it not support 
correctly stacked devices.
Its also possible that i dont understand how it should be configured or 
how it work :)

Please take a look on my problem/bug description, i think it could be a 
bug and in current versions it is not possible to create configuration 
with stacked devices.

Test env:
- 2 local servers a2 and a3, resource system-test
- 1 backup server z1 connected with a3, resource system-test-U
- resource system-test is up and primary on a3, secondary on a2, all 
UpToDate
- resource system-test-U is up and primary on a3, UpToDate
- resource system-test-U cannot be synchronized with z1 because a3 deny 
to enter SyncSource mode, drbdadm is executed without --stacked 
parameter and always return error with code 1

IMO problem is related to drbdadm command and lack of correct stacking 
support:
- when im trying to up resource on z1 i got dmesg messages on a3:
[1636671.252028] drbd system-test-U/0 drbd113 z1: helper command: 
/sbin/drbdadm before-resync-source
[1636671.255933] drbd system-test-U/0 drbd113: before-resync-source 
handler returned 1, dropping connection.
[1636671.255942] drbd system-test-U z1: conn( Connected -> Disconnecting 
) peer( Secondary -> Unknown )

- when i execute /sbin/drbdadm before-resync-source manually i got error 
message and error code 1
root at a3:/etc/drbd.d# /sbin/drbdadm before-resync-source system-test-U
'system-test-U' is a stacked resource, and not available in normal mode.

- when i execute it with --stacked parameter it works and got error code 0
root at a3:/etc/drbd.d# /sbin/drbdadm before-resync-source --stacked 
system-test-U

My configuration:
resource system-test {
  protocol C;
  device    /dev/drbd13;
  meta-disk internal;
  net {
   cram-hmac-alg "sha1";
   shared-secret "XXX";
  }
  on a2 {
   disk      /dev/vg_ssd/drbd-system-test;
   address   X.X.X.2:YY013;
  }
  on a3 {
   disk      /dev/vg_ssd/drbd-system-test;
   address   X.X.X.3:YY013;
  }
}
resource system-test-U {
  protocol A;
  device    /dev/drbd113;
  meta-disk internal;
  net {
   cram-hmac-alg "sha1";
   shared-secret "ZZZZZ";
  }
  stacked-on-top-of system-test {
   address   X.X.X.3:YY113;
  }
  on z1 {
   disk      /dev/vg_ssd/drbd-system-test;
   address   X.X.X.1:YY113;
  }
}

Im using Ubuntu 16.04 with drbd packages in versions:
ii  drbd-dkms 9.0.14-1ppa1~xenial1                       all          
RAID 1 over TCP/IP for Linux module source
ii  drbd-utils 9.4.0-1ppa1~xenial1                        amd64        
RAID 1 over TCP/IP for Linux (user utilities)

-- 
Artur Kaszuba