Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
It's not officially posted yet, but here is a link to a HOWTO that I
wrote, this should help :)
http://www.howtoforge.com/node/4265
: LINBIT | Your Way to High Availability
: 8152 SW Hall Blvd., Suite #209 : Beaverton, OR 97008
:
: http://www.linbit.com
DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
David.Livingstone at cn.ca wrote:
>
>
> Following on my Three node setup questions I have attempted to
> setup a three node cluster. See the drbd.conf at the end of this
> message and the following background.
>
> I get to the point of executing "drbdadm --stacked primary drbd-U"
> and I get the following error. Any help would be appreciated.
>
> [root at hatest1 etc]# drbdadm --stacked primary drbd-U
> /dev/drbd1: State change failed: (-2) Refusing to be Primary without at
> least one UpToDate disk
> Command 'drbdsetup /dev/drbd1 primary' terminated with exit code 17
>
>
> HW and SW Background :
>
> All three machines :
> - HW : - Proliant DL380G4
> - SW : - Latest RHEL5 and kernel-2.6.18-128.1.1.el5
> - drbd : drbd-8.3.0-3, drbd-km-2.6.18_128.1.1.el5-8.3.0-3
>
>
> Here is what the game plan was :
>
> 1. Bring up the underlying resource "drbd" :
> On hatest1/2:
> - drbdadm create-md drbd
> - modprobe drbd
> - drbdadm up drbd
> On hatest1:
> - drbdadm -- --overwrite-data-of-peer primary drbd
> 2. Manually on the "primary" bring up the 165.115.204.197(hatest)
> address specified for the stacked resource - it is this
> address which talks to the nlstor1 backup.
>
> On hatest1:
> ifconfig eth1:0 165.115.204.197 up
>
> 3. Start the stacked drbd-U resource :
>
> As with unstacked resources, you must create DRBD meta data on the
> stacked resources. This is done using the following command:
>
> On hatest1:
> drbdadm --stacked create-md drbd-U
>
> Then, you may enable the stacked resource:
>
> On hatest1:
> drbdadm --stacked up drbd-U
> drbdadm --stacked primary drbd-U
>
> After this, you may bring up the resource on the backup node, enabling
> three-node replication:
>
> On nlstor1:
> drbdadm create-md drbd-U
> drbdadm up drbd-U
>
> Here is the command history from the machines as they were executed :
>
> [root at hatest1 etc]# drbdadm create-md drbd
> md_offset 1572859904
> al_offset 1572827136
> bm_offset 1572777984
>
> Found some data
> ==> This might destroy existing data! <==
>
> Do you want to proceed?
> [need to type 'yes' to confirm] yes
>
> Writing meta data...
> initialising activity log
> NOT initialized bitmap
> New drbd meta data block sucessfully created.
> [root at hatest1 etc]#
>
> [root at hatest2 etc]# drbdadm create-md drbd
> md_offset 1572859904
> al_offset 1572827136
> bm_offset 1572777984
>
> Found some data
> ==> This might destroy existing data! <==
>
> Do you want to proceed?
> [need to type 'yes' to confirm] yes
>
> Writing meta data...
> initialising activity log
> NOT initialized bitmap
> New drbd meta data block sucessfully created.
> [root at hatest2 etc]#
>
> [root at hatest1 etc]# modprobe drbd
> [root at hatest1 etc]# lsmod|grep drbd
> drbd 220696 0
> [root at hatest1 etc]# service drbd status
> drbd driver loaded OK; device status:
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> m:res cs ro ds p mounted fstype
> [root at hatest1 etc]# drbdadm up drbd
> [root at hatest1 etc]#
>
> [root at hatest2 etc]# modprobe drbd
> [root at hatest2 etc]# service drbd status
> drbd driver loaded OK; device status:
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest2, 2009-02-13 09:59:05
> m:res cs ro ds p mounted fstype
> [root at hatest2 etc]# drbdadm up drbd
> [root at hatest2 etc]#
>
> [root at hatest1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r---
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:1535916
> [root at hatest1 etc]#
> [root at hatest2 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest2, 2009-02-13 09:59:05
> 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r---
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:1535916
> [root at hatest2 etc]#
>
> [root at hatest1 etc]# drbdadm -- --overwrite-data-of-peer primary drbd
> [root at hatest1 etc]#
>
> [root at hatest1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r---
> ns:1239832 nr:0 dw:0 dr:1241792 al:0 bm:74 lo:161 pe:175 ua:223 ap:0
> ep:1 wo:b oos:301676
> [===============>....] sync'ed: 80.6% (301676/1535916)K
> finish: 0:00:06 speed: 48,812 (49,368) K/sec
> [root at hatest1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
> ns:1535916 nr:0 dw:0 dr:1535916 al:0 bm:94 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b
> oos:0
> [root at hatest1 etc]#
>
> root at hatest2 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest2, 2009-02-13 09:59:05
> 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r---
> ns:0 nr:1535916 dw:1535916 dr:0 al:0 bm:94 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b
> oos:0
> [root at hatest2 etc]#
>
> [root at hatest1 etc]# drbdadm --stacked create-md drbd-U
> md_offset 1572773888
> al_offset 1572741120
> bm_offset 1572691968
>
> Found some data
> ==> This might destroy existing data! <==
>
> Do you want to proceed?
> [need to type 'yes' to confirm] yes
>
> Writing meta data...
> initialising activity log
> NOT initialized bitmap
> New drbd meta data block sucessfully created.
> [root at hatest1 etc]#
>
> [root at hatest1 etc]# drbdadm --stacked up drbd-U
> [root at hatest1 etc]#
>
> [root at hatest1 etc]# drbdadm --stacked primary drbd-U
> /dev/drbd1: State change failed: (-2) Refusing to be Primary without at
> least one UpToDate disk
> Command 'drbdsetup /dev/drbd1 primary' terminated with exit code 17
> [root at hatest1 etc]#
>
> [root at hatest1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
> ns:1536005 nr:0 dw:89 dr:1536107 al:1 bm:94 lo:0 pe:0 ua:0 ap:0 ep:1
> wo:b oos:0
> 1: cs:WFConnection ro:Secondary/Unknown ds:Inconsistent/DUnknown A r---
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:1535832
> [root at hatest1 etc]#
>
> root at nlstor1 etc]# drbdadm create-md drbd-U
> md_offset 2147479552
> al_offset 2147446784
> bm_offset 2147381248
>
> Found some data
> ==> This might destroy existing data! <==
>
> Do you want to proceed?
> [need to type 'yes' to confirm] yes
>
> Writing meta data...
> initialising activity log
> NOT initialized bitmap
> New drbd meta data block sucessfully created.
> [root at nlstor1 etc]#
>
> [root at nlstor1 etc]# drbdadm up drbd-U
> [root at nlstor1 etc]#
> [root at hatest1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at hatest1, 2009-02-13 09:00:29
> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
> ns:1536006 nr:0 dw:90 dr:1536158 al:1 bm:94 lo:0 pe:0 ua:0 ap:0 ep:1
> wo:b oos:0
> 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent A r---
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:1535832
> [root at hatest1 etc]#
> [root at nlstor1 etc]# cat /proc/drbd
> version: 8.3.0 (api:88/proto:86-89)
> GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by
> root at nlstor1, 2009-02-19 11:56:15
>
> 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent A r---
> ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:1535832
> [root at nlstor1 etc]#
>
> [root at hatest1 etc]# drbdadm --stacked primary drbd-U
> /dev/drbd1: State change failed: (-2) Refusing to be Primary without at
> least one UpToDate disk
> Command 'drbdsetup /dev/drbd1 primary' terminated with exit code 17
> [root at hatest1 etc]#
>
>
> Here is the drbd.conf :
>
>
>
> global {
> usage-count no;
> }
>
> common {
> disk {
> on-io-error detach; # What to do when the lower level device errors.
> }
>
> net {
> max-buffers 2048; #datablock buffers used before writing to disk.
> ko-count 4; # Peer is dead if this count is exceeded.
> #on-disconnect reconnect; # Peer disconnected, try to reconnect.
> }
>
> startup {
> wfc-timeout 0; # drbd init script will wait infinitely on resources.
> degr-wfc-timeout 120; # 2 minutes.
> }
> } # End of common
>
> resource drbd {
> protocol C;
> syncer {
> rate 100M; # Synchronization rate, in megebytes. Good for 100Mb network.
> al-extents 257; # Must be prime, number of active sets.
> }
> on hatest1 {
> device /dev/drbd0; # The name of our drbd device.
> disk /dev/hatest_vg/vol01; # Partition we wish drbd to use.
> address 192.168.36.130:7788; # node0 IP address and port number.
> meta-disk internal; # Stores meta-data in lower portion of disk.
> }
> on hatest2 {
> device /dev/drbd0; # Our drbd device, must match node0.
> disk /dev/hatest_vg/vol01; # Partition we wish drbd to use.
> address 192.168.36.129:7788; # node0 IP address and port number.
> meta-disk internal; # Stores meta-data in lower portion of disk.
> }
> } # End of resource drbd
>
> resource drbd-U {
> protocol A;
>
> syncer {
> rate 30M;
> al-extents 257;
> }
>
> stacked-on-top-of drbd {
> device /dev/drbd1;
> address 165.115.204.197:7788;
> }
>
> on nlstor1 {
> device /dev/drbd1; # The name of our drbd device.
> disk /dev/VolGroup00/LogVol02; # Partition we wish drbd to use.
> address 165.115.163.52:7788; # node0 IP address and port number.
> meta-disk internal; # Stores meta-data in lower portion of disk
> }
> } # End of resource drbd-U
>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user