Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
See below: >>> From: drbd-user-bounces at lists.linbit.com [mailto:drbd-user-bounces at lists.linbit.com] On Behalf Of FDS | Forensik Data Services Sent: Friday, September 06, 2013 9:06 AM To: drbd-user at lists.linbit.com Subject: [DRBD-user] generation identifiers Hi there, I am involved in the aftermath of a split brain event (drbd 8.3.8). log files have given us several entries for drbd_sync_handshakes: 1. Feb 28 07:10:04 san02 kernel: [ 1435.507176] block drbd1: self 384ACFA6DDE7F305:6EC706A11FB6AAEB:AD2868A6C65F20AA:D5016EA9D8E57175 bits:28397184 flags:0 2. Feb 28 07:10:04 san02 kernel: [ 1435.507472] block drbd0: self 8FFC8087A5A8ACDF:128A2EF7246101ED:00C719D52E8C2482:4EEC506B23021307 bits:225135 flags:0 3. Feb 28 07:11:36 san01 kernel: [4427691.562720] block drbd1: self 16DA13A883B368DA:6EC706A11FB6AAEA:AD2868A6C65F20AA:D5016EA9D8E57175 bits:3072 flags:0 4. Feb 28 07:11:36 san01 kernel: [4427691.562923] block drbd0: self 2DBF62FFC79BF9F4:128A2EF7246101EC:00C719D52E8C2483:4EEC506B23021307 bits:53722300 flags:0 5. Feb 28 07:17:46 san02 kernel: [ 1897.442652] block drbd0: self 8FFC8087A5A8ACDE:8A4BA5176FB14BAD:128A2EF7246101ED:00C719D52E8C2482 bits:21858 flags:0 6. Feb 28 07:17:46 san02 kernel: [ 1897.443239] block drbd1: self 384ACFA6DDE7F304:E9797096C25A015C:6EC706A11FB6AAEB:AD2868A6C65F20AA bits:24262845 flags:0 7. Feb 28 07:19:19 san01 kernel: [4428153.502129] block drbd0: self 8A4BA5176FB14BAC:0000000000000000:00C719D52E8C2483:4EEC506B23021307 bits:49588595 flags:0 8. Feb 28 07:19:19 san01 kernel: [4428153.502363] block drbd1: self E9797096C25A015C:0000000000000000:AD2868A6C65F20AA:D5016EA9D8E57175 bits:0 flags:0 The Cluster consists of 2 nodes (san01 und san02) with two block devices (drbd01 und drbd02) running in dual primary with OCFS2. Analyzing Generation Identifiers oft he first handshake: · Current UUIDs (1.-4.) differ due to split brainsituation. · BitMap UUIDs (1.-4.) are nearly identical. Only the last digit differs from A-D. First Historical UUIDs (1.-4.) with drbd1 are identical but differ for block drbd0. Again only the last digit (2 vs 3). · Second Historical UUID (1.-4.) for both block-Devices are identical meaning both devices have the same data set, or? Questions: · Reason and meaning for differing last digit UUIDs are unknown. No information in the doc. >>> I don't know · How can I see when the data sets were sync for the last time? >>> Careful analysis of the logs · Meaning of values for flags fe 0 or 2? >>> I don't know · Is it possible to decide which resource (drbd0 or drbd1) was primary by comparing UUIDs? >>> You had dual primary - they were both primary. You'll have to look at the data itself. For example if one file on one of the standalone drbd0 exists with a newer date, that file is likely more valid than the other file. Depending on how and why you had dual primary without proper fencing in the first place, it may be trivial to determine which to keep and which to "discard-my-data". If they are dual primary solely to provide live migration - AND - no live migrations were taking place at the time of the split brain event, you are fairly safe using the resource (or disk image) to which a VM was pointed at the time of the "event". If the file systems were shared by multiple hosts, you are stuck with file by file analysis - and there are no promises that the newest file will contain all the data. You may need to do a row by row analysis depending on your applications. >>> Again, without knowing the sharing and application logic in detail about your specific systems, split-brain analysis is impossible. >>> hth and good luck! >>> (Get fencing working!!!!!, Get a current drbd installed!!) >>> Dan Aprecciating your help. -- Mit freundlichen Grüßen Ing. Mag. Horst Greifeneder (fds) FDS | Forensik Data Services. Better secure than sorry! Schenkelbachweg 32. A - 4600 W e l s. tel. 07242. 777 15. fax. 07242. 777 16. mailto:office at fds.at -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130908/bc296280/attachment.htm>