[DRBD-user] Increasing a DRBD array

Gerald Brandt gbr at majentis.com
Sat Oct 22 15:33:15 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


Hi,



----- Original Message -----
> From: "Andreas Kurz" <andreas at hastexo.com>
> To: "drbd-user" <drbd-user at lists.linbit.com>
> Sent: Friday, October 21, 2011 6:12:11 PM
> Subject: Re: [DRBD-user] Increasing a DRBD array
> 
> On 10/21/2011 11:39 PM, Gerald Brandt wrote:
> > Hi,
> > 
> > I just saw that (google is my friend).  Can I change that on a
> > running drbd system?
> 
> hmm ... never tried changing meta-date that way ... shutdown,
> dump-md,
> reconfigure, create-md, restore-md might work ... maybe Lars has a
> hint ...
> 
> I would bring DRBD down on both nodes, stop it when all is in sync
> and
> recreate the meta data after changing the config and then skip the
> initial sync when bringing them up.
>

I really can't bring the nodes down.  I can bring down one at a time, but the systems have to stay running.
 
> > 
> > ie:
> > 
> > original:
> > 
> >         on iscsi-filer-1 {simply use "meta-disk internal;"
> >         device  /dev/drbd1;
> >         disk    /dev/md0;
> >         address 192.168.95.1:7789;
> >         meta-disk /dev/sda6[1];
> >         }
> > 
> >         on iscsi-filer-2 {
> >         device  /dev/drbd1;
> >         disk    /dev/md0;
> >         address 192.168.95.2:7789;
> >         meta-disk /dev/sda6[1];
> >         }
> > 
> > new:
> > 
> >         on iscsi-filer-1 {
> >         device  /dev/drbd1;
> >         disk    /dev/md0;
> >         address 192.168.95.1:7789;
> >         flexible-meta-disk /dev/sda6[1];
> 
> no ... that index thing only works for "static" meta-disk ... remove
> the
> [1] and resize /dev/sda6 if its not bigger than 196MB.

I'm not sure I understand.  /dev/sda6 is already 512 MB (I think).  Should I change to:

        on iscsi-filer-1 {
        device  /dev/drbd1;
        disk    /dev/md0;
        address 192.168.95.1:7789;
        flexible-meta-disk /dev/sda6;
        }

        on iscsi-filer-2 {
        device  /dev/drbd1;
        disk    /dev/md0;
        address 192.168.95.2:7789;
        flexible-meta-disk /dev/sda6;
        }

or would this be better:

        on iscsi-filer-1 {
        device  /dev/drbd1;
        disk    /dev/md0;
        address 192.168.95.1:7789;
        meta-disk internal;
        }

        on iscsi-filer-2 {
        device  /dev/drbd1;
        disk    /dev/md0;
        address 192.168.95.2:7789;
        meta-disk internal;
        }


I'll go back to my lists to see if I'm doing things right.

1. bring down the secondary
2. change the secondary to 'flexible-meta-data /dev/sda6' in drbd.conf on primary and secondary.
3. bring secondary back up (may re-sync entire disk, not a serious issue, just time)
4. repeat process for primary after re-sync (may cause another complete resync).

Gerald


> 
> Regards,
> Andreas
> 
> --
> Need help with DRBD?
> http://www.hastexo.com/now
> 
> >         }
> > 
> >         on iscsi-filer-2 {
> >         device  /dev/drbd1;
> >         disk    /dev/md0;
> >         address 192.168.95.2:7789;
> >         flexible-meta-disk /dev/sda6[1];
> >         }
> > 
> > Then reboot primary, followed by reboot secondary (after sync), and
> > all will be well?
> > 
> > Sorry if these seem to be noob questions.  I just want to be 100%
> > sure, as the file servers have live data on them.
> > 
> > Gerald
> > 
> > 
> > ----- Original Message -----
> >> From: "Andreas Kurz" <andreas at hastexo.com>
> >> To: drbd-user at lists.linbit.com
> >> Sent: Friday, October 21, 2011 4:26:13 PM
> >> Subject: Re: [DRBD-user] Increasing a DRBD array
> >>
> >> On 10/21/2011 10:48 PM, Gerald Brandt wrote:
> >>> Hi,
> >>>
> >>> DRBD is running directly on md0.  /dev/drbd1 is then exported via
> >>> iSCSI.
> >>>
> >>> The logs show:
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332010] drbd:
> >>> initialized.
> >>> Version: 8.3.7 (api:88/proto:86-91)
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332012] drbd:
> >>> GIT-hash:
> >>> ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by root at filer-1,
> >>> 2011-03-05 08:29:38
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332014] drbd:
> >>> registered as
> >>> block device major 147
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332015] drbd:
> >>> minor_table @
> >>> 0xffff88021dbaf300
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334144] block drbd1:
> >>> Starting worker thread (from cqueue [1489])
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334177] block drbd1:
> >>> ==>
> >>> truncating very big lower level device to currently maximum
> >>> possible
> >>> 8587575296 sectors <==
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334179] block drbd1:
> >>> ==>>
> >>> using internal or flexible meta data may help <<==
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334188] block drbd1:
> >>> disk(
> >>> Diskless -> Attaching )
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.353306] Loading
> >>> iSCSI
> >>> transport class v2.0-870.
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.360582] skge eth0:
> >>> disabling interface
> >>> Oct 21 15:34:53 iscsi-filer-1 iscsid: iSCSI logger with pid=1515
> >>> started!
> >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.381956] iscsi:
> >>> registered
> >>> transport (iser)
> >>> Oct 21 15:34:53 iscsi-filer-1 init: ssh main process (1162)
> >>> terminated
> >>> with status 255
> >>> Oct 21 15:34:53 iscsi-filer-1 postfix/master[1411]: reload --
> >>> version
> >>> 2.7.0, configuration /etc/postfix
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584678] block drbd1:
> >>> Found
> >>> 57 transactions (3507 active extents) in activity log.
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584684] block drbd1:
> >>> Method
> >>> to ensure write ordering: barrier
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584691] block drbd1:
> >>> Backing device's merge_bvec_fn() = ffffffffa00c0100
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584694] block drbd1:
> >>> max_segment_size ( = BIO size ) = 4096
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584698] block drbd1:
> >>> Adjusting my ra_pages to backing device's (32 -> 96)
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584704] block drbd1:
> >>> drbd_bm_resize called with capacity == 8587575296
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.603470] block drbd1:
> >>> resync
> >>> bitmap: bits=1073446912 words=16772608
> >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.603474] block drbd1:
> >>> size =
> >>> 4095 GB (4293787648 KB)
> >>>
> >>> The lines in yellow bug me.  I don't recall see them before.
> >>>
> >>> I had a 4 disk RAID-6 md0 (4x2TB = 4 TB RAID-6).  I added a
> >>> single
> >>> drive
> >>> (5x2TB = 6 TB array).
> >>
> >> Not using "meta-disk internal" or "flexible-meta-disk" limits the
> >> device
> >> size to 4TB (=128MB meta data size) ... change your metadata
> >> config
> >> ...
> >> as the logs suggest ... if you want to use all 6TB
> >>
> >> Regards,
> >> Andreas
> >>
> >> --
> >> Need help with DRBD?
> >> http://www.hastexo.com/now
> >>
> >>>
> >>> Any ideas?
> >>>
> >>> Gerald
> >>>
> >>>
> >>> ------------------------------------------------------------------------
> >>>
> >>>     From: "Andreas Kurz" <andreas at hastexo.com>
> >>>      To: drbd-user at lists.linbit.com
> >>>      Sent: Friday, October 21, 2011 2:55:06 PM
> >>>      Subject: Re: [DRBD-user] Increasing a DRBD array
> >>>      
> >>>      On 10/21/2011 09:30 PM, Gerald Brandt wrote:
> >>>      > Hi,
> >>>      >
> >>>      > I've successfully resize the lower level RAID-6 array, and
> >>>      > grown
> >>>      > it.  I'm now attempting to resize drbd, and nothing seems
> >>>      > to
> >>>      > happen.
> >>>      >
> >>>      > /dev/md0 is definitely bigger.
> >>>      >
> >>>      > What should I see during a drbd resize?
> >>>      
> >>>      You should see a DRBD resync of the newly added space.
> >>>      
> >>>      What is the lower level device of your DRBD resource? The
> >>>      whole md0,
> >>>      a
> >>>      partition on md0, a lv on a vg on a pv on md0?
> >>>      
> >>>      ... so if the lower level device has been resized on both
> >>>      nodes, DRBD
> >>>      should definitely grow on a "drbdadm resize".
> >>>      
> >>>      Did I mention that on starting DRBD it is resized
> >>>      automatically if a
> >>>      bigger lower level device is detected? ... have a look at
> >>>      the
> >>>      kernel
> >>>      logs...
> >>>      
> >>>      Regards,
> >>>      Andreas
> >>>      
> >>>      --
> >>>      Need help with DRBD?
> >>>      http://www.hastexo.com/now
> >>>      
> >>>      >
> >>>      > Gerald
> >>>      >
> >>>      >
> >>>      > ----- Original Message -----
> >>>      >> From: "Gerald Brandt" <gbr at majentis.com>
> >>>      >> To: drbd-user at lists.linbit.com
> >>>      >> Sent: Tuesday, October 18, 2011 7:08:55 AM
> >>>      >> Subject: Re: [DRBD-user] Increasing a DRBD array
> >>>      >>
> >>>      >> Hi,
> >>>      >>
> >>>      >> Okay, this is my list of what to do, and in what order:
> >>>      >>
> >>>      >> 1. remove the primary from DRBD
> >>>      >> 2. add the physical disk to the primary
> >>>      >> 3. add the primary back to DRBD and allow resync.
> >>>      >> 4. remove the secondary from DRBD
> >>>      >> 5. add the physical disk to the secondary
> >>>      >> 6. add the secondary back to DRBD and allow resync.
> >>>      >> 7. fdisk and add the disk to the RAID array on primary
> >>>      >> and
> >>>      >> secondary
> >>>      >> 8. grow the RAID array on the primary and secondary
> >>>      >> (while
> >>>      >> mounted
> >>>      >> or
> >>>      >> not mounted?)
> >>>      >> 9. drbdadm resize on the primary (can this be done while
> >>>      >> drbd is
> >>>      >> in
> >>>      >> use?)
> >>>      >> 10. expand the file system on the primary only (via
> >>>      >> Citrix
> >>>      >> XenServer
> >>>      >> (iSCSI))
> >>>      >>
> >>>      >> That sounds about right.  As long as I don't lose data.
> >>>      >> (yeah,
> >>>      >> backups.  That's left up to my client, but becomes my
> >>>      >> problem if
> >>>      >> they failed.)
> >>>      >>
> >>>      >> Gerald
> >>>      >>
> >>>      >>
> >>>      >>
> >>>      >> ----- Original Message -----
> >>>      >>> From: "Florian Haas" <florian at hastexo.com>
> >>>      >>> To: drbd-user at lists.linbit.com
> >>>      >>> Sent: Tuesday, October 18, 2011 3:53:11 AM
> >>>      >>> Subject: Re: [DRBD-user] Increasing a DRBD array
> >>>      >>>
> >>>      >>> On 2011-10-17 22:19, Matt Graham wrote:
> >>>      >>>>>> Your DRBD uses the MD RAID as its backing device,
> >>>      >>>>>> doesn't it?
> >>>      >>>>>> That means you'll have add the drive and resize your
> >>>      >>>>>> RAID
> >>>      >>>>>> first,
> >>>      >>>>>> then
> >>>      >>>>>> you can resize your DRBD, and then expand the file
> >>>      >>>>>> system.
> >>>      >>>>> Definitely built on top of the RAID.  I'm a bit
> >>>      >>>>> confused
> >>>      >>>>> by
> >>>      >>>>> your
> >>>      >>>>> answer though.  Step 8 grows the RAID array, and I
> >>>      >>>>> don't
> >>>      >>>>> have a
> >>>      >>>>> step
> >>>      >>>>> to grow DRBD.  Does that mean I'm missing something?
> >>>      >>>>
> >>>      >>>> 8. mdadm --grow /dev/md0 (other mdadm options)
> >>>      >>>>    Do that on both machines.
> >>>      >>>> 8.1 (on secondary) drbdsetup /dev/drbd0 resize
> >>>      >>>> 8.2 (on primary) drbdsetup /dev/drbd0 resize
> >>>      >>>
> >>>      >>> There's a "resize" subcommand for drbdadm too; no need
> >>>      >>> to
> >>>      >>> use
> >>>      >>> drbdsetup.
> >>>      >>> And it needs to be done on one node only. Just do
> >>>      >>> "drbdadm
> >>>      >>> resize
> >>>      >>> <resourcename>" on the primary while the two nodes are
> >>>      >>> connected,
> >>>      >>> and
> >>>      >>> you're good to go.
> >>>      >>>
> >>>      >>>>    The softRAID is below the DRBD device.  Once the
> >>>      >>>>    softRAID is
> >>>      >>>>    larger, the DRBD device needs to be informed of this
> >>>      >>>>    fact.
> >>>      >>>>    When making arrays/devices larger, you go bottom-up;
> >>>      >>>>    you make
> >>>      >>>>    the hardware changes first, then softRAID changes,
> >>>      >>>>    then DRBD
> >>>      >>>>    changes, then filesystem changes.
> >>>      >>>> 9. (on primary) resize2fs /dev/drbd0
> >>>      >>>
> >>>      >>> If it's ext3 and if it supports on-line resizing, of
> >>>      >>> course. But
> >>>      >>> that
> >>>      >>> should be obvious.
> >>>      >>>
> >>>      >>> Cheers,
> >>>      >>> Florian
> >>>      >>>
> >>>      >>> --
> >>>      >>> Need help with DRBD?
> >>>      >>> http://www.hastexo.com/knowledge/drbd
> >>>      >>> _______________________________________________
> >>>      >>> drbd-user mailing list
> >>>      >>> drbd-user at lists.linbit.com
> >>>      >>> http://lists.linbit.com/mailman/listinfo/drbd-user
> >>>      >>>
> >>>      >> _______________________________________________
> >>>      >> drbd-user mailing list
> >>>      >> drbd-user at lists.linbit.com
> >>>      >> http://lists.linbit.com/mailman/listinfo/drbd-user
> >>>      >>
> >>>      > _______________________________________________
> >>>      > drbd-user mailing list
> >>>      > drbd-user at lists.linbit.com
> >>>      > http://lists.linbit.com/mailman/listinfo/drbd-user
> >>>      
> >>>      
> >>>      
> >>>      
> >>>      
> >>>      _______________________________________________
> >>>      drbd-user mailing list
> >>>      drbd-user at lists.linbit.com
> >>>      http://lists.linbit.com/mailman/listinfo/drbd-user
> >>>      
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> drbd-user mailing list
> >>> drbd-user at lists.linbit.com
> >>> http://lists.linbit.com/mailman/listinfo/drbd-user
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> drbd-user mailing list
> >> drbd-user at lists.linbit.com
> >> http://lists.linbit.com/mailman/listinfo/drbd-user
> >>
> 
> 
> --
> Need help with DRBD?
> http://www.hastexo.com/now
> 
> 
> 
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
> 



More information about the drbd-user mailing list