Issue with Both Diskful Nodes Being Outdated in DRBD9

[Joel Colledge]

Hello Rui,

Thank you for the clear report.

> I am using DRBD 9.2.8

Please test again with DRBD 9.2.12. There have been some improvements
in this area since DRBD 9.2.8 such as:
44cb5fa46478 drbd: Avoid wrong outdates in far_away_change()

It is possible that the problem you have discovered is fixed by that
commit, or by another related one.

> 1. Is it possible to configure DRBD to disallow the promotion of an "Inconsistent" node to primary? This would help avoid this issue.

No, this is an important feature of DRBD. It should be possible to
promote a node whenever it has access to UpToDate data.

> 2. If both disked nodes are in the "outdated" state, is it guaranteed that their data is consistent? If the data is consistent, it would it be safe to use the --force option to promote one of the nodes to primary to resolve the situation.

Outdated data is always consistent. So yes, you can use "primary
--force" to promote one of the nodes. In this particular situation,
the data is actually up-to-date, so "primary --force" is the correct
way to fix the DRBD state. In other situations, using "primary
--force" on an Outdated node will cause divergent data (split-brain).

> 3. Can nodes in the "Inconsistent" or "Outdated" state participate in voting? Based on my understanding of distributed systems like etcd, unhealthy nodes are not allowed to vote or become leaders.

It's complicated. Explaining the details of the quorum algorithm is
beyond the scope of this message. For a discussion of some of the
details, see:
https://linbit.com/blog/drbd-quorum-implementation-updates/

Best regards,
Joel