[DRBD-user] Re: Problems with drbd on multipathed devices

Robert.Koeppl at knapp.com Robert.Koeppl at knapp.com
Mon Oct 12 18:43:24 CEST 2009

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Here are the config settings:
#multipaths {
#       #
#       # name  : multipath
#       # scope : multipath & multipathd
#       # desc  : container for settings that apply to one specific multipath
#       #
#       multipath {
#               #
#               # name  : wwid
#               # scope : multipath & multipathd
#               # desc  : index of the container
#               #
#               wwid                    3600508b4000156d700012000000b0000
#
#               #
#               # name    : alias
#               # scope   : multipath
#               # desc    : symbolic name for the multipath
#               #
#               alias                   yellow
#
#               #
#               # name    : path_grouping_policy
#               # scope   : multipath
#               # desc    : path grouping policy to apply to this multipath
#               # values  : failover, multibus, group_by_serial
#               # default : failover
#               #
#               path_grouping_policy    multibus
#
#               #
#               # name    : path_checker
#               # scope   : multipathd
#               # desc    : path checking alorithm to use to check path state
#               # values  : readsector0|tur|emc_clariion|hp_sw|directio
#               # default : directio
#               #
#               # path_checker          directio
#
#               #
#               # name    : path_selector
#               # desc    : the path selector algorithm to use for this mpath
#               #           these algo are offered by the kernel mpath target
#               # values  : "round-robin 0"
#               # default : "round-robin 0"
#               #
#               path_selector           "round-robin 0"
#
#               #
#               # name    : failback
#               # scope   : multipathd
#               # desc    : tell the daemon to manage path group failback, or not to.
#               #           0 means immediate failback, values >0 means deffered failback
#               #           expressed in seconds.
#               # values  : manual|immediate|n > 0
#               # default : manual
#               #
#               failback                immediate
#
#               #
#               # name    : no_path_retry
#               # scope   : multipath & multipathd
#               # desc    : tell the number of retries until disable queueing, or
#               #           "fail" means immediate failure (no queueing),
#               #           "queue" means never stop queueing
#               # values  : queue|fail|n (>0)
#               # default : (null)
#               #
#               #no_path_retry  queue
#
#               #
#               # name    : rr_min_io
#               # scope   : multipath
#               # desc    : the number of IO to route to a path before switching
#               #           to the next in the same path group
#               # default : 1000
#               #
#               rr_min_io       100
#
#               #
#               #name     : prio
#               #scope    : multipath
#               #desc     : the function to call to obtain a path weight.
#               #           Weights are summed for each path group to
#               #           determine the next PG to use case of failure.
#               #default : no callout, all paths equals
#               # Ex:
#               # prio  alua
#               # prio  "weightedpath devname sda 50 sde 10 sdc 50 sdf 10"
#               prio          "weightedpath hbtl 1:.:.:. 2 4:.:.:. 4"
#       }
#       multipath {
#               wwid    1DEC_____321816758474
#               alias   red
#               rr_weight               priorities
#       }
#}
#
##
## name  : devices
## scope : multipath & multipathd
## desc  : list of per storage controller settings
##        overrides default settings (device_maps block)
##         overriden by per multipath settings (multipaths block)
##
#devices {
#       #
#       # name  : device
#       # scope : multipath & multipathd
#       # desc  : settings for this specific storage controller
#       #
#       device {
#               #
#               # name  : vendor, product
#               # scope : multipath & multipathd
#               # desc  : index for the block
#               #
#               vendor                  "COMPAQ  "
#               product                 "HSV110 (C)COMPAQ"
#
#               #
#               # name    : path_grouping_policy
#               # scope   : multipath
#               # desc    : path grouping policy to apply to multipath hosted
#               #           by this storage controller
#               # values  : failover        = 1 path per priority group
#               #           multibus        = all valid paths in 1 priority
#               #                             group
#               #           group_by_serial = 1 priority group per detected
#               #                             serial number
#               # default : failover
#               #
#               path_grouping_policy    multibus
#
#               #
#               # name    : getuid_callout
#               # scope   : multipath
#               # desc    : the program and args to callout to obtain a unique
#               #           path identifier. Absolute path required
#               # default : /lib/udev/scsi_id -g -u -s
#               #
#               getuid_callout          "/lib/udev/scsi_id -g -u -d /dev/%n"
#
#               #
#               # name    : prio
#               # scope   : multipath
#               # desc    : the function to call to obtain a path
#               #           weight. Weights are summed for each path group to
#               #           determine the next PG to use case of failure.
#               # default : no callout, all paths equals
#               #
#               prio          "hp_sw"
#
#               #
#               # name    : path_checker
#               # scope   : multipathd
#               # desc    : path checking alorithm to use to check path state
#               # values  : readsector0|tur|emc_clariion|hp_sw|directio
#               # default : directio
#               #
#               path_checker            directio
#
#               #
#               # name    : path_selector
#               # desc    : the path selector algorithm to use for this mpath
#               #           these algo are offered by the kernel mpath target
#               # values  : "round-robin 0"
#               # default : "round-robin 0"
#               #
#               path_selector           "round-robin 0"
#
#               #
#               # name    : failback
#               # scope   : multipathd
#               # desc    : tell the daemon to manage path group failback, or not to.
#               #           0 means immediate failback, values >0 means deffered failback
#               #           expressed in seconds.
#               # values  : manual|immediate|n > 0
#               # default : manual
#               #
#               failback                30
#
#               #
#               # name    : rr_min_io
#               # scope   : multipath
#               # desc    : the number of IO to route to a path before switching
#               #           to the next in the same path group
#               # default : 1000
#               #
#               rr_min_io       100
#
#               #
#               # name    : product_blacklist
#               # scope   : multipath & multipathd
#               # desc    : product strings to blacklist for this vendor
#               # default : none
#               #
#               product_blacklist       LUN_Z
#       }
#       device {
#               vendor                  "COMPAQ  "
#               product                 "MSA1000         "
#               path_grouping_policy    multibus
#               path_checker            tur
#               rr_weight               priorities
#       }
#}
device {
                vendor "IBM"
                product "1814"
                path_grouping_policy "group_by_prio"
                path_checker "rdac"
                hardware_handler "1 rdac"
                prio "rdac"
                no_path_retry "fail"
        }


k1348kc1:~ #


  }

  on k1348kc2 {
    device     /dev/drbd2;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000005794aa90388;
    address    192.168.1.2:7792;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[2];      
  }
}
resource r3 {


  on k1348kc1 {
    device     /dev/drbd3;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a464aa954d1;
    address    192.168.1.1:7793;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[3];        
  }

  on k1348kc2 {
    device     /dev/drbd3;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000005834aa90689;
    address    192.168.1.2:7793;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[3];      
  }
}
resource r4 {


  on k1348kc1 {
    device     /dev/drbd4;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a424aa9545f;
    address    192.168.1.1:7794;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[4];        
  }

  on k1348kc2 {
    device     /dev/drbd4;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e0000057f4aa9046c;
    address    192.168.1.2:7795;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[4];      
  }
}
resource r5 {


  on k1348kc1 {
    device     /dev/drbd5;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a404aa95417;
    address    192.168.1.1:7796;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[5];        
  }

  on k1348kc2 {
    device     /dev/drbd5;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e0000057d4aa9041e;
    address    192.168.1.2:7796;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[5];      
  }
}
resource r6 {


  on k1348kc1 {
    device     /dev/drbd6;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a3e4aa953cf;
    address    192.168.1.1:7797;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[6];        
  }

  on k1348kc2 {
    device     /dev/drbd6;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e0000057b4aa903cb;
    address    192.168.1.2:7797;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[6];      
  }
}
resource r7 {


  on k1348kc1 {
    device     /dev/drbd7;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a444aa95491;
    address    192.168.1.1:7798;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[7];        
  }

  on k1348kc2 {
    device     /dev/drbd7;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000005814aa904be;
    address    192.168.1.2:7798;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[7];      
  }
}


resource r8 {


  on k1348kc1 {
    device     /dev/drbd8;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a374aa9524c;
    address    192.168.1.1:7799;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[8];        
  }

  on k1348kc2 {
    device     /dev/drbd8;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000005744aa9027c;
    address    192.168.1.2:7799;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[8];      
  }
}

resource r9 {


  on k1348kc1 {
    device     /dev/drbd9;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000a2a4aa95041;
    address    192.168.1.1:7800;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[9];        
  }

  on k1348kc2 {
    device     /dev/drbd9;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000005724aa90249;
    address    192.168.1.2:7800;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[9];      
  }
}

resource r10 {


  on k1348kc1 {
    device     /dev/drbd10;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000c814ac1ecd0;
    address    192.168.1.1:7801;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[10];      
  }

  on k1348kc2 {
    device     /dev/drbd10;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e0000087d4ac1ef7f;
    address    192.168.1.2:7801;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[10];      
  }
}
resource r11 {


  on k1348kc1 {
    device     /dev/drbd11;
    disk       /dev/disk/by-id/scsi-3600a0b800050cabc000008d74ac1ec3d;
    address    192.168.1.1:7802;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[11];      
  }

  on k1348kc2 {
    device     /dev/drbd11;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc8800000a0e4ac1ef8c;
    address    192.168.1.2:7802;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[11];      
  }
}
resource r12 {


  on k1348kc1 {
    device     /dev/drbd12;
    disk       /dev/disk/by-id/scsi-3600a0b800050cb2000000c844ac1ed09;
    address    192.168.1.1:7803;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[12];      
  }

  on k1348kc2 {
    device     /dev/drbd12;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc6e000008804ac1efae;
    address    192.168.1.2:7803;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[12];      
  }
}
resource r13 {


  on k1348kc1 {
    device     /dev/drbd13;
    disk       /dev/disk/by-id/scsi-3600a0b800050cabc000008d94ac1ec97;
    address    192.168.1.1:7804;
   meta-disk  /dev/disk/by-id/scsi-3600a0b800050cb2000000a274aa94ffe[13];      
  }

  on k1348kc2 {
    device     /dev/drbd13;
    disk       /dev/disk/by-id/scsi-3600a0b800050dc8800000a104ac1efb6;
    address    192.168.1.2:7804;
    meta-disk  /dev/disk/by-id/scsi-3600a0b800050dc6e000005694aa900a0[13];      
  }
}
k1348kc1:~ #


Here is the output of multipath -ll
k1348kc1:~ # multipath -ll
3600a0b800050cb2000000c814ac1ecd0 dm-11 IBM,1814      FAStT
[size=5.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:12 sdm  8:192  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:12 sdab 65:176 [active][ghost]
3600a0b800050cb2000000a3c4aa9537d dm-4 IBM,1814      FAStT
[size=40G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:5  sdf  8:80   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:5  sdu  65:64  [active][ghost]
3600a0b800050cb2000000a3a4aa95340 dm-3 IBM,1814      FAStT
[size=10G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:4  sde  8:64   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:4  sdt  65:48  [active][ghost]
3600a0b800050cb2000000c844ac1ed09 dm-13 IBM,1814      FAStT
[size=5.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:14 sdo  8:224  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:14 sdad 65:208 [active][ghost]
3600a0b800050cb2000000a374aa9524c dm-2 IBM,1814      FAStT
[size=100G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:3  sdd  8:48   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:3  sds  65:32  [active][ghost]
3600a0b800050cb2000000a424aa9545f dm-6 IBM,1814      FAStT
[size=40G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:7  sdh  8:112  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:7  sdw  65:96  [active][ghost]
3600a0b800050cabc000008d94ac1ec97 dm-14 IBM,1814      FAStT
[size=5.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 4:0:0:15 sdae 65:224 [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 3:0:0:15 sdp  8:240  [active][ghost]
3600a0b800050cb2000000a274aa94ffe dm-10 IBM,1814      FAStT
[size=3.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:11 sdl  8:176  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:11 sdaa 65:160 [active][ghost]
3600a0b800050cb2000000a464aa954d1 dm-5 IBM,1814      FAStT
[size=20G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:6  sdg  8:96   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:6  sdv  65:80  [active][ghost]
3600a0b800050cb2000000a404aa95417 dm-7 IBM,1814      FAStT
[size=20G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:8  sdi  8:128  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:8  sdx  65:112 [active][ghost]
3600a0b800050cabc000008d74ac1ec3d dm-12 IBM,1814      FAStT
[size=5.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 4:0:0:13 sdac 65:192 [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 3:0:0:13 sdn  8:208  [active][ghost]
3600a0b800050cb2000000a2c4aa95089 dm-0 IBM,1814      FAStT
[size=200G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:1  sdb  8:16   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:1  sdq  65:0   [active][ghost]
3600a0b800050cb2000000a444aa95491 dm-9 IBM,1814      FAStT
[size=1.0G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:10 sdk  8:160  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:10 sdz  65:144 [active][ghost]
3600a0b800050cb2000000a3e4aa953cf dm-8 IBM,1814      FAStT
[size=20G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:9  sdj  8:144  [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:9  sdy  65:128 [active][ghost]
3600a0b800050cb2000000a2a4aa95041 dm-1 IBM,1814      FAStT
[size=100G][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=6][active]
 \_ 3:0:0:2  sdc  8:32   [active][ready]
\_ round-robin 0 [prio=1][enabled]
 \_ 4:0:0:2  sdr  65:16  [active][ghost]
k1348kc1:~ #

Mit freundlichen Grüßen / Best regards,

Robert Köppl



More information about the drbd-user mailing list