[DRBD-user] Increasing a DRBD array

Andreas Kurz andreas at hastexo.com
Sat Oct 22 01:12:11 CEST 2011

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


On 10/21/2011 11:39 PM, Gerald Brandt wrote:
> Hi,
> 
> I just saw that (google is my friend).  Can I change that on a running drbd system?

hmm ... never tried changing meta-date that way ... shutdown, dump-md,
reconfigure, create-md, restore-md might work ... maybe Lars has a hint ...

I would bring DRBD down on both nodes, stop it when all is in sync and
recreate the meta data after changing the config and then skip the
initial sync when bringing them up.

> 
> ie:
> 
> original:
> 
>         on iscsi-filer-1 {simply use "meta-disk internal;"
>         device  /dev/drbd1;
>         disk    /dev/md0;
>         address 192.168.95.1:7789;
>         meta-disk /dev/sda6[1];
>         }
> 
>         on iscsi-filer-2 {
>         device  /dev/drbd1;
>         disk    /dev/md0;
>         address 192.168.95.2:7789;
>         meta-disk /dev/sda6[1];
>         }
> 
> new:
> 
>         on iscsi-filer-1 {
>         device  /dev/drbd1;
>         disk    /dev/md0;
>         address 192.168.95.1:7789;
>         flexible-meta-disk /dev/sda6[1];

no ... that index thing only works for "static" meta-disk ... remove the
[1] and resize /dev/sda6 if its not bigger than 196MB.

Regards,
Andreas

-- 
Need help with DRBD?
http://www.hastexo.com/now

>         }
> 
>         on iscsi-filer-2 {
>         device  /dev/drbd1;
>         disk    /dev/md0;
>         address 192.168.95.2:7789;
>         flexible-meta-disk /dev/sda6[1];
>         }
> 
> Then reboot primary, followed by reboot secondary (after sync), and all will be well?
> 
> Sorry if these seem to be noob questions.  I just want to be 100% sure, as the file servers have live data on them.
> 
> Gerald
> 
> 
> ----- Original Message -----
>> From: "Andreas Kurz" <andreas at hastexo.com>
>> To: drbd-user at lists.linbit.com
>> Sent: Friday, October 21, 2011 4:26:13 PM
>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>
>> On 10/21/2011 10:48 PM, Gerald Brandt wrote:
>>> Hi,
>>>
>>> DRBD is running directly on md0.  /dev/drbd1 is then exported via
>>> iSCSI.
>>>
>>> The logs show:
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332010] drbd:
>>> initialized.
>>> Version: 8.3.7 (api:88/proto:86-91)
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332012] drbd:
>>> GIT-hash:
>>> ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by root at filer-1,
>>> 2011-03-05 08:29:38
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332014] drbd:
>>> registered as
>>> block device major 147
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.332015] drbd:
>>> minor_table @
>>> 0xffff88021dbaf300
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334144] block drbd1:
>>> Starting worker thread (from cqueue [1489])
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334177] block drbd1:
>>> ==>
>>> truncating very big lower level device to currently maximum
>>> possible
>>> 8587575296 sectors <==
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334179] block drbd1:
>>> ==>>
>>> using internal or flexible meta data may help <<==
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.334188] block drbd1:
>>> disk(
>>> Diskless -> Attaching )
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.353306] Loading iSCSI
>>> transport class v2.0-870.
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.360582] skge eth0:
>>> disabling interface
>>> Oct 21 15:34:53 iscsi-filer-1 iscsid: iSCSI logger with pid=1515
>>> started!
>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [    7.381956] iscsi:
>>> registered
>>> transport (iser)
>>> Oct 21 15:34:53 iscsi-filer-1 init: ssh main process (1162)
>>> terminated
>>> with status 255
>>> Oct 21 15:34:53 iscsi-filer-1 postfix/master[1411]: reload --
>>> version
>>> 2.7.0, configuration /etc/postfix
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584678] block drbd1:
>>> Found
>>> 57 transactions (3507 active extents) in activity log.
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584684] block drbd1:
>>> Method
>>> to ensure write ordering: barrier
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584691] block drbd1:
>>> Backing device's merge_bvec_fn() = ffffffffa00c0100
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584694] block drbd1:
>>> max_segment_size ( = BIO size ) = 4096
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584698] block drbd1:
>>> Adjusting my ra_pages to backing device's (32 -> 96)
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.584704] block drbd1:
>>> drbd_bm_resize called with capacity == 8587575296
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.603470] block drbd1:
>>> resync
>>> bitmap: bits=1073446912 words=16772608
>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [    7.603474] block drbd1:
>>> size =
>>> 4095 GB (4293787648 KB)
>>>
>>> The lines in yellow bug me.  I don't recall see them before.
>>>
>>> I had a 4 disk RAID-6 md0 (4x2TB = 4 TB RAID-6).  I added a single
>>> drive
>>> (5x2TB = 6 TB array).
>>
>> Not using "meta-disk internal" or "flexible-meta-disk" limits the
>> device
>> size to 4TB (=128MB meta data size) ... change your metadata config
>> ...
>> as the logs suggest ... if you want to use all 6TB
>>
>> Regards,
>> Andreas
>>
>> --
>> Need help with DRBD?
>> http://www.hastexo.com/now
>>
>>>
>>> Any ideas?
>>>
>>> Gerald
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>     From: "Andreas Kurz" <andreas at hastexo.com>
>>>      To: drbd-user at lists.linbit.com
>>>      Sent: Friday, October 21, 2011 2:55:06 PM
>>>      Subject: Re: [DRBD-user] Increasing a DRBD array
>>>      
>>>      On 10/21/2011 09:30 PM, Gerald Brandt wrote:
>>>      > Hi,
>>>      >
>>>      > I've successfully resize the lower level RAID-6 array, and
>>>      > grown
>>>      > it.  I'm now attempting to resize drbd, and nothing seems to
>>>      > happen.
>>>      >
>>>      > /dev/md0 is definitely bigger.
>>>      >
>>>      > What should I see during a drbd resize?
>>>      
>>>      You should see a DRBD resync of the newly added space.
>>>      
>>>      What is the lower level device of your DRBD resource? The
>>>      whole md0,
>>>      a
>>>      partition on md0, a lv on a vg on a pv on md0?
>>>      
>>>      ... so if the lower level device has been resized on both
>>>      nodes, DRBD
>>>      should definitely grow on a "drbdadm resize".
>>>      
>>>      Did I mention that on starting DRBD it is resized
>>>      automatically if a
>>>      bigger lower level device is detected? ... have a look at the
>>>      kernel
>>>      logs...
>>>      
>>>      Regards,
>>>      Andreas
>>>      
>>>      --
>>>      Need help with DRBD?
>>>      http://www.hastexo.com/now
>>>      
>>>      >
>>>      > Gerald
>>>      >
>>>      >
>>>      > ----- Original Message -----
>>>      >> From: "Gerald Brandt" <gbr at majentis.com>
>>>      >> To: drbd-user at lists.linbit.com
>>>      >> Sent: Tuesday, October 18, 2011 7:08:55 AM
>>>      >> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>      >>
>>>      >> Hi,
>>>      >>
>>>      >> Okay, this is my list of what to do, and in what order:
>>>      >>
>>>      >> 1. remove the primary from DRBD
>>>      >> 2. add the physical disk to the primary
>>>      >> 3. add the primary back to DRBD and allow resync.
>>>      >> 4. remove the secondary from DRBD
>>>      >> 5. add the physical disk to the secondary
>>>      >> 6. add the secondary back to DRBD and allow resync.
>>>      >> 7. fdisk and add the disk to the RAID array on primary and
>>>      >> secondary
>>>      >> 8. grow the RAID array on the primary and secondary (while
>>>      >> mounted
>>>      >> or
>>>      >> not mounted?)
>>>      >> 9. drbdadm resize on the primary (can this be done while
>>>      >> drbd is
>>>      >> in
>>>      >> use?)
>>>      >> 10. expand the file system on the primary only (via Citrix
>>>      >> XenServer
>>>      >> (iSCSI))
>>>      >>
>>>      >> That sounds about right.  As long as I don't lose data.
>>>      >> (yeah,
>>>      >> backups.  That's left up to my client, but becomes my
>>>      >> problem if
>>>      >> they failed.)
>>>      >>
>>>      >> Gerald
>>>      >>
>>>      >>
>>>      >>
>>>      >> ----- Original Message -----
>>>      >>> From: "Florian Haas" <florian at hastexo.com>
>>>      >>> To: drbd-user at lists.linbit.com
>>>      >>> Sent: Tuesday, October 18, 2011 3:53:11 AM
>>>      >>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>      >>>
>>>      >>> On 2011-10-17 22:19, Matt Graham wrote:
>>>      >>>>>> Your DRBD uses the MD RAID as its backing device,
>>>      >>>>>> doesn't it?
>>>      >>>>>> That means you'll have add the drive and resize your
>>>      >>>>>> RAID
>>>      >>>>>> first,
>>>      >>>>>> then
>>>      >>>>>> you can resize your DRBD, and then expand the file
>>>      >>>>>> system.
>>>      >>>>> Definitely built on top of the RAID.  I'm a bit confused
>>>      >>>>> by
>>>      >>>>> your
>>>      >>>>> answer though.  Step 8 grows the RAID array, and I don't
>>>      >>>>> have a
>>>      >>>>> step
>>>      >>>>> to grow DRBD.  Does that mean I'm missing something?
>>>      >>>>
>>>      >>>> 8. mdadm --grow /dev/md0 (other mdadm options)
>>>      >>>>    Do that on both machines.
>>>      >>>> 8.1 (on secondary) drbdsetup /dev/drbd0 resize
>>>      >>>> 8.2 (on primary) drbdsetup /dev/drbd0 resize
>>>      >>>
>>>      >>> There's a "resize" subcommand for drbdadm too; no need to
>>>      >>> use
>>>      >>> drbdsetup.
>>>      >>> And it needs to be done on one node only. Just do "drbdadm
>>>      >>> resize
>>>      >>> <resourcename>" on the primary while the two nodes are
>>>      >>> connected,
>>>      >>> and
>>>      >>> you're good to go.
>>>      >>>
>>>      >>>>    The softRAID is below the DRBD device.  Once the
>>>      >>>>    softRAID is
>>>      >>>>    larger, the DRBD device needs to be informed of this
>>>      >>>>    fact.
>>>      >>>>    When making arrays/devices larger, you go bottom-up;
>>>      >>>>    you make
>>>      >>>>    the hardware changes first, then softRAID changes,
>>>      >>>>    then DRBD
>>>      >>>>    changes, then filesystem changes.
>>>      >>>> 9. (on primary) resize2fs /dev/drbd0
>>>      >>>
>>>      >>> If it's ext3 and if it supports on-line resizing, of
>>>      >>> course. But
>>>      >>> that
>>>      >>> should be obvious.
>>>      >>>
>>>      >>> Cheers,
>>>      >>> Florian
>>>      >>>
>>>      >>> --
>>>      >>> Need help with DRBD?
>>>      >>> http://www.hastexo.com/knowledge/drbd
>>>      >>> _______________________________________________
>>>      >>> drbd-user mailing list
>>>      >>> drbd-user at lists.linbit.com
>>>      >>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>      >>>
>>>      >> _______________________________________________
>>>      >> drbd-user mailing list
>>>      >> drbd-user at lists.linbit.com
>>>      >> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>      >>
>>>      > _______________________________________________
>>>      > drbd-user mailing list
>>>      > drbd-user at lists.linbit.com
>>>      > http://lists.linbit.com/mailman/listinfo/drbd-user
>>>      
>>>      
>>>      
>>>      
>>>      
>>>      _______________________________________________
>>>      drbd-user mailing list
>>>      drbd-user at lists.linbit.com
>>>      http://lists.linbit.com/mailman/listinfo/drbd-user
>>>      
>>>
>>>
>>>
>>> _______________________________________________
>>> drbd-user mailing list
>>> drbd-user at lists.linbit.com
>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>
>>
>>
>>
>> _______________________________________________
>> drbd-user mailing list
>> drbd-user at lists.linbit.com
>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>


-- 
Need help with DRBD?
http://www.hastexo.com/now


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 286 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20111022/239db1d1/attachment.pgp>


More information about the drbd-user mailing list