Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi, Unfortunately, I didn't find the time to test with DRBD 8.4.2 again so far. But I sadly encountered the same bug again in DRBD 8.4.3 during our Ganeti stress tests. We have a DRBD volume on top of two LV's (one for the data, the other for metadata). Xen uses this volume for a VM which gets migrated to the secondary node in our test. Therefore, Ganeti switches the DRBD volume to dual-primary mode (on the secondary node), promotes the secondary node to primary node and tells Xen to migrate the running instance to the (old) secondary node. Unfortunately, somewhere in the process DRBD looses synchronization with a ProtocolError and fails to reconnect, as this would create a split-brain situation. So, as cool as it would have been, we won't be able to take advantage of the on-the-fly changing into dual-master mode and back. Instead, we will continue as with DRBD 8.3.x and detach the volume, change to dual-master mode, reattach and resync, and then do the same reversed after the instance migration... Thanks, Thomas On Mon, May 6, 2013 at 8:22 AM, Thomas Thrainer <thomasth at google.com> wrote: > Hi, > > I'm was running the vanilla Wheezy kernel for my DRBD 8.4.3 tests. If I > get to it, I'll try DRBD 8.4.2 again with that kernel. > It would still be interesting if this issues is/was already known and > "officially" fixed in DRBD 8.4.3 (the release notes don't mention it), as I > can't quite test all possible combinations between kernels and DRBD > versions. > > Cheers, > Thomas > > > On Fri, May 3, 2013 at 6:24 PM, Louis <mailtolouis2020-forum at yahoo.com>wrote: > >> Hi Thomas, >> >> Why don't you re-test DRBD 8.4.2 using Wheezy (Linux 3.2.0-4-amd64 #1 SMP >> Debian 3.2.41-2 x86_64 GNU/Linux) ? If the error show again, that could be >> the answer that the problem is fixed in 8.4.3. >> >> >> Regards >> Louis >> >> ------------------------------ >> *From:* Thomas Thrainer <thomasth at google.com> >> *To:* Louis <mailtolouis2020-forum at yahoo.com> >> *Cc:* Luca Fornasari <luca.fornasari at gmail.com>; DRBD < >> drbd-user at lists.linbit.com> >> *Sent:* Friday, May 3, 2013 2:14 PM >> >> *Subject:* Re: [DRBD-user] Enabling two primaries break connection >> >> Hi, >> >> I just re-tested using Wheezy (Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2 >> x86_64 GNU/Linux) and DRBD 8.4.3. >> I couldn't reproduce the original bug (having one side writing, and >> toggling the other side between secondary/primary). However, if I try to >> write on the "toggling" side to the disk when in primary mode, I could >> still produce ProtocolError's. >> As that's (fortunately) not required by Ganeti, we'll can probably ignore >> this (although I will test some more of our use-cases with DRBD 8.4.3). >> >> Can somebody confirm that the original behavior was a bug in DRBD 8.4.2 >> which was fixed in DRBD 8.4.3? Or could interactions with the kernel be the >> problem? >> >> Thanks, >> Thomas >> >> >> On Fri, May 3, 2013 at 2:56 PM, Louis <mailtolouis2020-forum at yahoo.com>wrote: >> >> >> >> Linux intel1 3.5.0-23-generic #35~precise1-Ubuntu SMP >> >> >> ------------------------------ >> *From:* Luca Fornasari <luca.fornasari at gmail.com> >> *To:* DRBD <drbd-user at lists.linbit.com> >> *Sent:* Friday, May 3, 2013 1:01 PM >> >> *Subject:* Re: [DRBD-user] Enabling two primaries break connection >> >> Hi Louis, >> >> What kernel are yuo running? >> >> Cheers, >> Luca >> >> >> On Fri, May 3, 2013 at 1:41 PM, Louis <mailtolouis2020-forum at yahoo.com>wrote: >> >> Hi Thomas, >> >> I just tested your scenario on DRBD 8.4.3. It seems fine, connection did >> not break. >> While doing the DD, I did change primary / secondary on bob node few >> times, but the connection never break. >> >> The only thing I couldn't run this drbdadm --allow-two-primaries=yes >> <resource>, it give uknown command. But I do have allow-two-primaries setup >> in my drbd.conf, not sure that make any different? >> >> Regards >> Louis >> >> ------------------------------ >> *From:* Thomas Thrainer <thomasth at google.com> >> *To:* Luca Fornasari <luca.fornasari at gmail.com> >> *Cc:* drbd-user at lists.linbit.com >> *Sent:* Friday, May 3, 2013 10:00 AM >> *Subject:* Re: [DRBD-user] Enabling two primaries break connection >> >> Hi, >> >> Given the fact that I could reproduce the bug regularly, I'm pretty sure >> that this is indeed a DRBD bug. Unfortunately I didn't find a bugtracker on >> the DRBD site. Is there one? If not, what's the official way to report bugs? >> >> For the record, here are the steps to reproduce: >> * create a device with primary on alice and secondary on bob >> * on alice: while true; do dd if=/dev/zero of=/dev/drbd0; done >> * on bob: drbdadm --allow-two-primaries=yes <resource>; drbdadm primary >> <resource> >> * you might have to drbdadm secondary <resource> on bob and then repeat >> the previous step in order to produce the bug >> >> Log files are as mentioned in the first mail of the thread. >> >> Cheers, >> Thomas >> >> >> >> On Tue, Apr 30, 2013 at 2:25 PM, Thomas Thrainer <thomasth at google.com>wrote: >> >> Hi Luca, >> >> I just reproduced the exact same problem using only drbdadm. I also >> inspected the output of the -d option, and drbdadm does the exact same >> thing as Ganeti in this regard. >> >> Thanks, >> Thomas >> >> >> On Tue, Apr 30, 2013 at 2:18 PM, Thomas Thrainer <thomasth at google.com>wrote: >> >> Hi Luca, >> >> I'll try the drbdadm method this as soon as I get to it. Right now I have >> no machines around which are already configured via drbdadm. >> >> Note that I have included both logs (secondary and primary) in my first >> email. >> >> I use DRBD 8.4.2 because 8.4.3 didn't compile against my kernel (3.2.27) >> and I didn't want to put much work into it. I this bug is known and fixed >> in 8.4.3, I'll might look into compiling against 3.2.27 again. >> >> Cheers, >> Thomas >> >> -- >> Thomas Thrainer | Software Engineer | thomasth at google.com | >> >> Google Germany GmbH >> Dienerstr. 12 >> 80331 München >> >> Registergericht und -nummer: Hamburg, HRB 86891 >> Sitz der Gesellschaft: Hamburg >> Geschäftsführer: Graham Law, Katherine Stephens >> >> >> >> >> -- >> Thomas Thrainer | Software Engineer | thomasth at google.com | >> >> Google Germany GmbH >> Dienerstr. 12 >> 80331 München >> >> Registergericht und -nummer: Hamburg, HRB 86891 >> Sitz der Gesellschaft: Hamburg >> Geschäftsführer: Graham Law, Katherine Stephens >> >> >> >> >> -- >> Thomas Thrainer | Software Engineer | thomasth at google.com | >> >> Google Germany GmbH >> Dienerstr. 12 >> 80331 München >> >> Registergericht und -nummer: Hamburg, HRB 86891 >> Sitz der Gesellschaft: Hamburg >> Geschäftsführer: Graham Law, Katherine Stephens >> >> _______________________________________________ >> drbd-user mailing list >> drbd-user at lists.linbit.com >> http://lists.linbit.com/mailman/listinfo/drbd-user >> >> >> >> _______________________________________________ >> drbd-user mailing list >> drbd-user at lists.linbit.com >> http://lists.linbit.com/mailman/listinfo/drbd-user >> >> >> >> _______________________________________________ >> drbd-user mailing list >> drbd-user at lists.linbit.com >> http://lists.linbit.com/mailman/listinfo/drbd-user >> >> >> >> _______________________________________________ >> drbd-user mailing list >> drbd-user at lists.linbit.com >> http://lists.linbit.com/mailman/listinfo/drbd-user >> >> >> >> >> -- >> Thomas Thrainer | Software Engineer | thomasth at google.com | >> >> Google Germany GmbH >> Dienerstr. 12 >> 80331 München >> >> Registergericht und -nummer: Hamburg, HRB 86891 >> Sitz der Gesellschaft: Hamburg >> Geschäftsführer: Graham Law, Katherine Stephens >> >> >> >> _______________________________________________ >> drbd-user mailing list >> drbd-user at lists.linbit.com >> http://lists.linbit.com/mailman/listinfo/drbd-user >> >> > > > -- > Thomas Thrainer | Software Engineer | thomasth at google.com | > > Google Germany GmbH > Dienerstr. 12 > 80331 München > > Registergericht und -nummer: Hamburg, HRB 86891 > Sitz der Gesellschaft: Hamburg > Geschäftsführer: Graham Law, Katherine Stephens > -- Thomas Thrainer | Software Engineer | thomasth at google.com | Google Germany GmbH Dienerstr. 12 80331 München Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschäftsführer: Graham Law, Katherine Stephens -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130507/5fa7aa44/attachment.htm>