[Drbd-dev] [RFC] (CRM and) DRBD (0.8) states and transistions, recovery strategies

[Philipp Reisner]

On Wednesday 29 September 2004 19:07, Lars Ellenberg wrote:
> / 2004-09-29 14:58:47 +0200
>
> \ Philipp Reisner:
> > > [...]
> > >
> > > >     Currently this covers only the states, and outlines the
> > > > transitions. It should help to define the actions to be taken on
> > > > every possible "input" to the DRBD internal "state machine".
> > >
> > > While reading through this giant e-mail I lost my confidence that it
> > > could be a good idea to have a "central" state switching function in
> > > DRBD, but of course I will see what this discussions gives...
> >
> > Thought about this a bit more... and came to the conclusion that it
> > would be a good idea. What do you think of this skeleton -
> > pseude code (it compiles actually).
>
> I think it probably should come out more like a real state machine,
> with a defined set of possible INPUTS,
> a defined set of states (which should not have the same detail depth as
> the actual drbd internal state set with all its different attributes),
> a set of actions, and a defined state[INPUT] => action => newstate
> matrix.
>
> maybe that is overkill.

Currently I want to go the way that was outlined with the new_st.c
skeleton.

Regarding: only worker should do state changes.
  We have quite a lot of inputs that are asynchronous by their nature.
  E.g. Disk fails. It does not make any sense to synchronize an advance
  in the disk-state state-machine with anything.

While it makes a lot of sense to synchronize changes to the node
state machine. 

At first I drew a directed graph of the cstates we have in 
drbd-0.7 (see cstates-7.ps)

You will immediately realize that the differentiation between
Unconfigured and StandAllone is a leftover from drbd-0.6

Then I drew directed graphs of the "state machines" as I see
them for drbd-0.8

conn-states-8.ps, disk-states-8.ps, node-states-8.ps (has 2 pages)

PS: The program is graphviz

-Philipp
-- 
: Dipl-Ing Philipp Reisner                      Tel +43-1-8178292-50 :
: LINBIT Information Technologies GmbH          Fax +43-1-8178292-82 :
: Schönbrunnerstr 244, 1120 Vienna, Austria    http://www.linbit.com :
-------------- next part --------------
digraph conn_states {
	StandAllone  -> WFConnection   [ label = "ioctl_set_net()" ]
	WFConnection -> Unconnected    [ label = "unable to bind()" ]
	WFConnection -> WFReportParams [ label = "in connect() after accept" ]
	WFReportParams -> StandAllone  [ label = "checks in receive_param()" ]
	WFReportParams -> Connected    [ label = "in receive_param()" ]
	WFReportParams -> WFBitMapS    [ label = "sync_handshake()" ]
	WFReportParams -> WFBitMapT    [ label = "sync_handshake()" ]
	WFBitMapS -> SyncSource        [ label = "receive_bitmap()" ]
	WFBitMapT -> SyncTarget        [ label = "receive_bitmap()" ]
	SyncSource -> Connected
	SyncTarget -> Connected
	SyncSource -> PausedSyncS
	SyncTarget -> PausedSyncT
	PausedSyncS -> SyncSource
	PausedSyncT -> SyncTarget
	Connected   -> WFConnection    [ label = "* on network error" ]
}
-------------- next part --------------
digraph disk_states {
	Diskless -> Inconsistent       [ label = "ioctl_set_disk()" ]
	Diskless -> Consistent         [ label = "ioctl_set_disk()" ]
	Diskless -> Outdated           [ label = "ioctl_set_disk()" ]
	Consistent -> Outdated         [ label = "receive_param()" ]
	Consistent -> UpToDate         [ label = "receive_param()" ]
	Consistent -> Inconsistent     [ label = "start resync" ]
	Outdated   -> Inconsistent     [ label = "start resync" ]
	UpToDate   -> Inconsistent     [ label = "ioctl_replicate" ]
	Inconsistent -> UpToDate       [ label = "resync completed" ]
	Consistent -> Failed           [ label = "io completion error" ]
	Outdated   -> Failed           [ label = "io completion error" ]
	UpToDate   -> Failed           [ label = "io completion error" ]
	Inconsistent -> Failed         [ label = "io completion error" ]
	Failed -> Diskless             [ label = "sending notify to peer" ]
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: disk-states-8.ps
Type: application/postscript
Size: 12540 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-dev/attachments/20041006/a3b1c5ab/disk-states-8-0001.ps
-------------- next part --------------
digraph node_states {
	Secondary -> Primary           [ label = "ioctl_set_state()" ]
	Primary   -> Secondary 	       [ label = "ioctl_set_state()" ]
}

digraph peer_states {
	Secondary -> Primary           [ label = "recv state packet" ]
	Primary   -> Secondary 	       [ label = "recv state packet" ]
	Primary   -> Unknown 	       [ label = "connection lost" ]
	Secondary  -> Unknown  	       [ label = "connection lost" ]
	Unknown   -> Primary           [ label = "connected" ]
	Unknown   -> Secondary         [ label = "connected" ]
}

-------------- next part --------------
A non-text attachment was scrubbed...
Name: node-states-8.ps
Type: application/postscript
Size: 9774 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-dev/attachments/20041006/a3b1c5ab/node-states-8-0001.ps
-------------- next part --------------
digraph cstate {
	Unconfigured -> StandAllone    [ label = "ioctl_set_disk()" ]
	StandAllone  -> Unconnected    [ label = "ioctl_set_net()" ]
	Unconfigured -> Unconnected    [ label = "ioctl_set_net()" ]
	Unconnected  -> WFConnection   [ label = "connect()[1]" ]
	WFConnection -> Unconnected    [ label = "unable to bind()" ]
	WFConnection -> WFReportParams [ label = "in connect() after accept" ]
	WFReportParams -> StandAllone  [ label = "checks in receive_param()" ]
	WFReportParams -> Connected    [ label = "in receive_param()" ]
	WFReportParams -> WFBitMapS    [ label = "sync_handshake()" ]
	WFReportParams -> WFBitMapT    [ label = "sync_handshake()" ]
	WFBitMapS -> SyncSource        [ label = "receive_bitmap()" ]
	WFBitMapT -> SyncTarget        [ label = "receive_bitmap()" ]
	SyncSource -> Connected
	SyncTarget -> Connected
	SyncSource -> PausedSyncS
	SyncTarget -> PausedSyncT
	PausedSyncS -> SyncSource
	PausedSyncT -> SyncTarget
	Connected   -> BrokenPipe       [ label = "* recv error" ]
	BrokenPipe  -> WFConnection     [ label = "connect()[1]" ]
	Connected   -> NetworkFailure   [ label = "* set by asender()" ]
	NetworkFailure -> WFConnection  [ label = "connect()[1]" ]
	Connected   -> Timeout          [ label = "* set drbd_send()" ]
	Timeout     -> WFConnection     [ label = "connect()[1]" ]
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: conn-states-8.ps
Type: application/postscript
Size: 13380 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-dev/attachments/20041006/a3b1c5ab/conn-states-8-0001.ps
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cstates-7.ps
Type: application/postscript
Size: 17743 bytes
Desc: not available
Url : http://lists.linbit.com/pipermail/drbd-dev/attachments/20041006/a3b1c5ab/cstates-7-0001.ps