Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Philipp Reisner wrote: > But, we are very interested in how you got DRBD into that. We want to > reproduce that there. Could you provide more context please ? > > Is it dom0 or DomU ? > How many cores ? > Logfile context (~20 Lines) before and after the start of the > online verify, from both nodes. > > Thanks! > Hi Phil, will try to give you as much info as needed. Two node XEN/Heartbeat cluster (nodes ajax and ariel). Both systems run Debian Lenny 2.6.26-1-xen-686, h/w differs, but each has 1 hyperthreaded CPU, seen as 2 CPU's. DRBD resources in Dom0's sit on LVM LV's, one resource per LV. On top of this are DomU's, one per cluster node. I am attaching drbd.conf, maybe you find it useful. And last messages from DRBD. Actually there are no more messages since verify run. Should you need more info, I will be glad to help. Software is the same on both ariel and ajax so I am listing only one node here (ajax). Kind regards, Ivars ajax:/var/log# dpkg -l | grep -i drbd ii drbd8-utils 2:8.3.0-1 RAID 1 over tcp/ip for Linux utilities ajax:/var/log# ajax:~# /etc/init.d/drbd status drbd driver loaded OK; device status: version: 8.3.0 (api:88/proto:86-89) GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by root at xen-test, 2009-02-02 17:16:36 m:res cs ro ds p mounted fstype 0:proxyrootlv Connected Primary/Secondary UpToDate/UpToDate C 1:proxydatalv Connected Primary/Secondary UpToDate/UpToDate C 2:proxyswaplv Connected Primary/Secondary UpToDate/UpToDate C 3:mailrootlv Connected Secondary/Primary UpToDate/UpToDate C 4:mailvarlv Connected Secondary/Primary UpToDate/UpToDate C 5:mailhomelv Connected Secondary/Primary UpToDate/UpToDate C 6:mailswaplv Connected Secondary/Primary UpToDate/UpToDate C ajax:~# Log on ajax: ajax:/var/log# zless kern.log.1.gz | tail -n 50 Mar 27 18:27:25 ajax kernel: [277417.632534] drbd2: role( Primary -> Secondary ) Mar 27 18:27:25 ajax kernel: [277417.944774] drbd0: role( Primary -> Secondary ) Mar 27 18:27:26 ajax kernel: [277419.050930] drbd2: role( Secondary -> Primary ) Mar 27 18:27:27 ajax kernel: [277419.174372] device vif13.0 entered promiscuous mode Mar 27 18:27:27 ajax kernel: [277419.222957] drbd0: role( Secondary -> Primary ) Mar 27 18:27:27 ajax kernel: [277419.224288] xenbr0: port 2(vif13.0) entering learning state Mar 27 18:27:27 ajax kernel: [277419.407818] drbd1: role( Secondary -> Primary ) Mar 27 18:27:28 ajax kernel: [277420.642634] blkback: ring-ref 8, event-channel 9, protocol 1 (x86_32-abi) Mar 27 18:27:28 ajax kernel: [277420.696951] blkback: ring-ref 9, event-channel 10, protocol 1 (x86_32-abi) Mar 27 18:27:28 ajax kernel: [277420.750011] blkback: ring-ref 10, event-channel 11, protocol 1 (x86_32-abi) Mar 27 18:27:37 ajax kernel: [277430.200506] vif13.0: no IPv6 routers present Mar 27 18:27:42 ajax kernel: [277434.349017] xenbr0: topology change detected, propagating Mar 27 18:27:42 ajax kernel: [277434.388307] xenbr0: port 2(vif13.0) entering forwarding state Mar 27 18:57:05 ajax kernel: [279199.711270] drbd4: peer( Primary -> Secondary ) Mar 27 18:57:05 ajax kernel: [279199.784485] drbd5: peer( Primary -> Secondary ) Mar 27 18:57:05 ajax kernel: [279199.826808] drbd6: peer( Primary -> Secondary ) Mar 27 18:57:05 ajax kernel: [279199.882700] drbd3: peer( Primary -> Secondary ) Mar 27 18:57:07 ajax kernel: [279201.504502] drbd3: peer( Secondary -> Primary ) Mar 27 18:57:07 ajax kernel: [279201.660454] drbd4: peer( Secondary -> Primary ) Mar 27 18:57:07 ajax kernel: [279201.711044] drbd6: peer( Secondary -> Primary ) Mar 27 18:57:07 ajax kernel: [279201.866755] drbd5: peer( Secondary -> Primary ) Mar 27 20:15:39 ajax kernel: [283919.703323] drbd6: peer( Primary -> Secondary ) Mar 27 20:15:39 ajax kernel: [283919.742341] drbd3: peer( Primary -> Secondary ) Mar 27 20:15:39 ajax kernel: [283919.783014] drbd4: peer( Primary -> Secondary ) Mar 27 20:15:39 ajax kernel: [283919.876850] drbd5: peer( Primary -> Secondary ) Mar 27 20:15:41 ajax kernel: [283921.518474] drbd6: peer( Secondary -> Primary ) Mar 27 20:15:41 ajax kernel: [283921.667995] drbd5: peer( Secondary -> Primary ) Mar 27 20:15:41 ajax kernel: [283921.710328] drbd3: peer( Secondary -> Primary ) Mar 27 20:15:41 ajax kernel: [283921.747597] drbd4: peer( Secondary -> Primary ) Mar 29 00:42:02 ajax kernel: [386416.997962] drbd0: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.038717] drbd1: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.120563] drbd2: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.360109] drbd3: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.475596] drbd4: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.640808] drbd5: conn( Connected -> VerifyS ) Mar 29 00:42:02 ajax kernel: [386417.713500] drbd6: conn( Connected -> VerifyS ) Mar 29 00:42:57 ajax kernel: [386473.477001] drbd1: Online verify done (total 55 sec; paused 0 sec; 9532 K/sec) Mar 29 00:42:57 ajax kernel: [386473.513222] drbd1: conn( VerifyS -> Connected ) Mar 29 00:43:44 ajax kernel: [386521.967747] drbd2: Online verify done (total 102 sec; paused 0 sec; 10280 K/sec) Mar 29 00:43:44 ajax kernel: [386522.009362] drbd2: conn( VerifyS -> Connected ) Mar 29 00:43:44 ajax kernel: [386522.159134] drbd6: Online verify done (total 102 sec; paused 0 sec; 10280 K/sec) Mar 29 00:43:44 ajax kernel: [386522.198502] drbd6: conn( VerifyS -> Connected ) Mar 29 00:47:11 ajax kernel: [386732.857289] drbd0: Online verify done (total 309 sec; paused 0 sec; 13572 K/sec) Mar 29 00:47:11 ajax kernel: [386732.901375] drbd0: conn( VerifyS -> Connected ) Mar 29 00:48:02 ajax kernel: [386785.773514] drbd3: Online verify done (total 360 sec; paused 0 sec; 14560 K/sec) Mar 29 00:48:02 ajax kernel: [386785.773514] drbd3: conn( VerifyS -> Connected ) Mar 29 00:51:24 ajax kernel: [386992.870891] drbd4: Online verify done (total 561 sec; paused 0 sec; 18688 K/sec) Mar 29 00:51:24 ajax kernel: [386992.919033] drbd4: conn( VerifyS -> Connected ) Mar 29 01:00:30 ajax kernel: [387555.388163] drbd5: Online verify done (total 1107 sec; paused 0 sec; 28416 K/sec) Mar 29 01:00:30 ajax kernel: [387555.435757] drbd5: conn( VerifyS -> Connected ) ariel:/var/log# zless kern.log.1.gz | tail -n 50 Mar 27 18:57:10 ariel kernel: [241217.958095] blkback: ring-ref 10, event-channel 11, protocol 1 (x86_32-abi) Mar 27 18:57:10 ariel kernel: [241217.970943] blkback: ring-ref 11, event-channel 12, protocol 1 (x86_32-abi) Mar 27 18:57:18 ariel kernel: [241225.818573] vif5.0: no IPv6 routers present Mar 27 18:57:22 ariel kernel: [241230.244991] xenbr0: topology change detected, propagating Mar 27 18:57:22 ariel kernel: [241230.245006] xenbr0: port 2(vif5.0) entering forwarding state Mar 27 20:15:38 ariel kernel: [245931.886027] xenbr0: port 2(vif5.0) entering disabled state Mar 27 20:15:38 ariel kernel: [245931.892205] xenbr0: port 2(vif5.0) entering disabled state Mar 27 20:15:39 ariel kernel: [245933.112451] drbd6: role( Primary -> Secondary ) Mar 27 20:15:39 ariel kernel: [245933.140366] drbd3: role( Primary -> Secondary ) Mar 27 20:15:39 ariel kernel: [245933.185212] drbd4: role( Primary -> Secondary ) Mar 27 20:15:39 ariel kernel: [245933.262990] drbd5: role( Primary -> Secondary ) Mar 27 20:15:41 ariel kernel: [245934.989750] drbd6: role( Secondary -> Primary ) Mar 27 20:15:41 ariel kernel: [245935.023563] device vif6.0 entered promiscuous mode Mar 27 20:15:41 ariel kernel: [245935.037677] xenbr0: port 2(vif6.0) entering learning state Mar 27 20:15:41 ariel kernel: [245935.134388] drbd5: role( Secondary -> Primary ) Mar 27 20:15:41 ariel kernel: [245935.173874] drbd3: role( Secondary -> Primary ) Mar 27 20:15:41 ariel kernel: [245935.214646] drbd4: role( Secondary -> Primary ) Mar 27 20:15:44 ariel kernel: [245937.677838] blkback: ring-ref 8, event-channel 9, protocol 1 (x86_32-abi) Mar 27 20:15:44 ariel kernel: [245937.696405] blkback: ring-ref 9, event-channel 10, protocol 1 (x86_32-abi) Mar 27 20:15:44 ariel kernel: [245937.707687] blkback: ring-ref 10, event-channel 11, protocol 1 (x86_32-abi) Mar 27 20:15:44 ariel kernel: [245937.718321] blkback: ring-ref 11, event-channel 12, protocol 1 (x86_32-abi) Mar 27 20:15:52 ariel kernel: [245945.940261] vif6.0: no IPv6 routers present Mar 27 20:15:56 ariel kernel: [245950.092923] xenbr0: topology change detected, propagating Mar 27 20:15:56 ariel kernel: [245950.092932] xenbr0: port 2(vif6.0) entering forwarding state Mar 29 00:42:01 ariel kernel: [348441.934901] drbd0: conn( Connected -> VerifyT ) Mar 29 00:42:01 ariel kernel: [348441.974093] drbd1: conn( Connected -> VerifyT ) Mar 29 00:42:02 ariel kernel: [348442.052935] drbd2: conn( Connected -> VerifyT ) Mar 29 00:42:02 ariel kernel: [348442.295670] drbd3: conn( Connected -> VerifyT ) Mar 29 00:42:02 ariel kernel: [348442.410327] drbd4: conn( Connected -> VerifyT ) Mar 29 00:42:02 ariel kernel: [348442.575957] drbd5: conn( Connected -> VerifyT ) Mar 29 00:42:02 ariel kernel: [348442.646426] drbd6: conn( Connected -> VerifyT ) Mar 29 00:42:57 ariel kernel: [348497.895760] drbd1: Online verify done (total 55 sec; paused 0 sec; 9532 K/sec) Mar 29 00:42:57 ariel kernel: [348497.895780] drbd1: conn( VerifyT -> Connected ) Mar 29 00:43:00 ariel kernel: [348501.157876] drbd2: in got_OVResult:4043: rs_pending_cnt = -1 < 0 ! Mar 29 00:43:44 ariel kernel: [348545.960264] drbd2: Online verify done (total 102 sec; paused 0 sec; 10280 K/sec) Mar 29 00:43:44 ariel kernel: [348545.960264] drbd2: conn( VerifyT -> Connected ) Mar 29 00:43:44 ariel kernel: [348546.067889] drbd6: Online verify done (total 102 sec; paused 0 sec; 10280 K/sec) Mar 29 00:43:44 ariel kernel: [348546.067907] drbd6: conn( VerifyT -> Connected ) Mar 29 00:44:55 ariel kernel: [348617.260443] drbd4: in got_OVResult:4043: rs_pending_cnt = -1 < 0 ! Mar 29 00:46:11 ariel kernel: [348693.863181] drbd3: in got_OVResult:4043: rs_pending_cnt = -1 < 0 ! Mar 29 00:46:25 ariel kernel: [348708.441138] drbd5: in got_OVResult:4043: rs_pending_cnt = -1 < 0 ! Mar 29 00:46:56 ariel kernel: [348739.932875] drbd3: in got_OVResult:4043: rs_pending_cnt = -1 < 0 ! Mar 29 00:47:11 ariel kernel: [348754.216463] drbd0: Online verify done (total 309 sec; paused 0 sec; 13572 K/sec) Mar 29 00:47:11 ariel kernel: [348754.216463] drbd0: conn( VerifyT -> Connected ) Mar 29 00:48:02 ariel kernel: [348806.450389] drbd3: Online verify done (total 360 sec; paused 0 sec; 14560 K/sec) Mar 29 00:48:02 ariel kernel: [348806.450403] drbd3: conn( VerifyT -> Connected ) Mar 29 00:51:24 ariel kernel: [349011.096385] drbd4: Online verify done (total 561 sec; paused 0 sec; 18688 K/sec) Mar 29 00:51:24 ariel kernel: [349011.096385] drbd4: conn( VerifyT -> Connected ) Mar 29 01:00:30 ariel kernel: [349566.866018] drbd5: Online verify done (total 1107 sec; paused 0 sec; 28416 K/sec) Mar 29 01:00:30 ariel kernel: [349566.866034] drbd5: conn( VerifyT -> Connected ) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: drbd.conf URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20090330/1b62d07f/attachment.asc>