Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Okay, this is the plan for changing an primary/secondary drbd from 'meta-disk /dev/sda6[1] to 'flexible-meta-disk /dev/sda6'
sda6 is already 512 MB in size, which in theory will let me go to 16 TB storage (128 MB is 4 TB storage).
on secondary:
1. drbdadm down iscsi.target.0
2. edit drbd.conf to reflect meta-disk change (also on primary?)
3. drbdadm create-md iscsi.target.0
4. drbdadm up iscsi.target.0
5. wait for full sync to take place
on primary
6-10 same as 1 - 5 above
At with point I should be able to do drbdadm resize iscsi.target.0, and I should see 6TB of storage.
That that sound right to everyone? It should give me no downtime and the ability to have more than 4TB storage.
Gerald
----- Original Message -----
> From: "Gerald Brandt" <gbr at majentis.com>
> To: "Andreas Kurz" <andreas at hastexo.com>
> Cc: "drbd-user" <drbd-user at lists.linbit.com>
> Sent: Saturday, October 22, 2011 8:33:15 AM
> Subject: Re: [DRBD-user] Increasing a DRBD array
>
> Hi,
>
>
>
> ----- Original Message -----
> > From: "Andreas Kurz" <andreas at hastexo.com>
> > To: "drbd-user" <drbd-user at lists.linbit.com>
> > Sent: Friday, October 21, 2011 6:12:11 PM
> > Subject: Re: [DRBD-user] Increasing a DRBD array
> >
> > On 10/21/2011 11:39 PM, Gerald Brandt wrote:
> > > Hi,
> > >
> > > I just saw that (google is my friend). Can I change that on a
> > > running drbd system?
> >
> > hmm ... never tried changing meta-date that way ... shutdown,
> > dump-md,
> > reconfigure, create-md, restore-md might work ... maybe Lars has a
> > hint ...
> >
> > I would bring DRBD down on both nodes, stop it when all is in sync
> > and
> > recreate the meta data after changing the config and then skip the
> > initial sync when bringing them up.
> >
>
> I really can't bring the nodes down. I can bring down one at a time,
> but the systems have to stay running.
>
> > >
> > > ie:
> > >
> > > original:
> > >
> > > on iscsi-filer-1 {simply use "meta-disk internal;"
> > > device /dev/drbd1;
> > > disk /dev/md0;
> > > address 192.168.95.1:7789;
> > > meta-disk /dev/sda6[1];
> > > }
> > >
> > > on iscsi-filer-2 {
> > > device /dev/drbd1;
> > > disk /dev/md0;
> > > address 192.168.95.2:7789;
> > > meta-disk /dev/sda6[1];
> > > }
> > >
> > > new:
> > >
> > > on iscsi-filer-1 {
> > > device /dev/drbd1;
> > > disk /dev/md0;
> > > address 192.168.95.1:7789;
> > > flexible-meta-disk /dev/sda6[1];
> >
> > no ... that index thing only works for "static" meta-disk ...
> > remove
> > the
> > [1] and resize /dev/sda6 if its not bigger than 196MB.
>
> I'm not sure I understand. /dev/sda6 is already 512 MB (I think).
> Should I change to:
>
> on iscsi-filer-1 {
> device /dev/drbd1;
> disk /dev/md0;
> address 192.168.95.1:7789;
> flexible-meta-disk /dev/sda6;
> }
>
> on iscsi-filer-2 {
> device /dev/drbd1;
> disk /dev/md0;
> address 192.168.95.2:7789;
> flexible-meta-disk /dev/sda6;
> }
>
> or would this be better:
>
> on iscsi-filer-1 {
> device /dev/drbd1;
> disk /dev/md0;
> address 192.168.95.1:7789;
> meta-disk internal;
> }
>
> on iscsi-filer-2 {
> device /dev/drbd1;
> disk /dev/md0;
> address 192.168.95.2:7789;
> meta-disk internal;
> }
>
>
> I'll go back to my lists to see if I'm doing things right.
>
> 1. bring down the secondary
> 2. change the secondary to 'flexible-meta-data /dev/sda6' in
> drbd.conf on primary and secondary.
> 3. bring secondary back up (may re-sync entire disk, not a serious
> issue, just time)
> 4. repeat process for primary after re-sync (may cause another
> complete resync).
>
> Gerald
>
>
> >
> > Regards,
> > Andreas
> >
> > --
> > Need help with DRBD?
> > http://www.hastexo.com/now
> >
> > > }
> > >
> > > on iscsi-filer-2 {
> > > device /dev/drbd1;
> > > disk /dev/md0;
> > > address 192.168.95.2:7789;
> > > flexible-meta-disk /dev/sda6[1];
> > > }
> > >
> > > Then reboot primary, followed by reboot secondary (after sync),
> > > and
> > > all will be well?
> > >
> > > Sorry if these seem to be noob questions. I just want to be 100%
> > > sure, as the file servers have live data on them.
> > >
> > > Gerald
> > >
> > >
> > > ----- Original Message -----
> > >> From: "Andreas Kurz" <andreas at hastexo.com>
> > >> To: drbd-user at lists.linbit.com
> > >> Sent: Friday, October 21, 2011 4:26:13 PM
> > >> Subject: Re: [DRBD-user] Increasing a DRBD array
> > >>
> > >> On 10/21/2011 10:48 PM, Gerald Brandt wrote:
> > >>> Hi,
> > >>>
> > >>> DRBD is running directly on md0. /dev/drbd1 is then exported
> > >>> via
> > >>> iSCSI.
> > >>>
> > >>> The logs show:
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332010] drbd:
> > >>> initialized.
> > >>> Version: 8.3.7 (api:88/proto:86-91)
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332012] drbd:
> > >>> GIT-hash:
> > >>> ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by root at filer-1,
> > >>> 2011-03-05 08:29:38
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332014] drbd:
> > >>> registered as
> > >>> block device major 147
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332015] drbd:
> > >>> minor_table @
> > >>> 0xffff88021dbaf300
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334144] block
> > >>> drbd1:
> > >>> Starting worker thread (from cqueue [1489])
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334177] block
> > >>> drbd1:
> > >>> ==>
> > >>> truncating very big lower level device to currently maximum
> > >>> possible
> > >>> 8587575296 sectors <==
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334179] block
> > >>> drbd1:
> > >>> ==>>
> > >>> using internal or flexible meta data may help <<==
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334188] block
> > >>> drbd1:
> > >>> disk(
> > >>> Diskless -> Attaching )
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.353306] Loading
> > >>> iSCSI
> > >>> transport class v2.0-870.
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.360582] skge eth0:
> > >>> disabling interface
> > >>> Oct 21 15:34:53 iscsi-filer-1 iscsid: iSCSI logger with
> > >>> pid=1515
> > >>> started!
> > >>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.381956] iscsi:
> > >>> registered
> > >>> transport (iser)
> > >>> Oct 21 15:34:53 iscsi-filer-1 init: ssh main process (1162)
> > >>> terminated
> > >>> with status 255
> > >>> Oct 21 15:34:53 iscsi-filer-1 postfix/master[1411]: reload --
> > >>> version
> > >>> 2.7.0, configuration /etc/postfix
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584678] block
> > >>> drbd1:
> > >>> Found
> > >>> 57 transactions (3507 active extents) in activity log.
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584684] block
> > >>> drbd1:
> > >>> Method
> > >>> to ensure write ordering: barrier
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584691] block
> > >>> drbd1:
> > >>> Backing device's merge_bvec_fn() = ffffffffa00c0100
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584694] block
> > >>> drbd1:
> > >>> max_segment_size ( = BIO size ) = 4096
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584698] block
> > >>> drbd1:
> > >>> Adjusting my ra_pages to backing device's (32 -> 96)
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584704] block
> > >>> drbd1:
> > >>> drbd_bm_resize called with capacity == 8587575296
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.603470] block
> > >>> drbd1:
> > >>> resync
> > >>> bitmap: bits=1073446912 words=16772608
> > >>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.603474] block
> > >>> drbd1:
> > >>> size =
> > >>> 4095 GB (4293787648 KB)
> > >>>
> > >>> The lines in yellow bug me. I don't recall see them before.
> > >>>
> > >>> I had a 4 disk RAID-6 md0 (4x2TB = 4 TB RAID-6). I added a
> > >>> single
> > >>> drive
> > >>> (5x2TB = 6 TB array).
> > >>
> > >> Not using "meta-disk internal" or "flexible-meta-disk" limits
> > >> the
> > >> device
> > >> size to 4TB (=128MB meta data size) ... change your metadata
> > >> config
> > >> ...
> > >> as the logs suggest ... if you want to use all 6TB
> > >>
> > >> Regards,
> > >> Andreas
> > >>
> > >> --
> > >> Need help with DRBD?
> > >> http://www.hastexo.com/now
> > >>
> > >>>
> > >>> Any ideas?
> > >>>
> > >>> Gerald
> > >>>
> > >>>
> > >>> ------------------------------------------------------------------------
> > >>>
> > >>> From: "Andreas Kurz" <andreas at hastexo.com>
> > >>> To: drbd-user at lists.linbit.com
> > >>> Sent: Friday, October 21, 2011 2:55:06 PM
> > >>> Subject: Re: [DRBD-user] Increasing a DRBD array
> > >>>
> > >>> On 10/21/2011 09:30 PM, Gerald Brandt wrote:
> > >>> > Hi,
> > >>> >
> > >>> > I've successfully resize the lower level RAID-6 array,
> > >>> > and
> > >>> > grown
> > >>> > it. I'm now attempting to resize drbd, and nothing
> > >>> > seems
> > >>> > to
> > >>> > happen.
> > >>> >
> > >>> > /dev/md0 is definitely bigger.
> > >>> >
> > >>> > What should I see during a drbd resize?
> > >>>
> > >>> You should see a DRBD resync of the newly added space.
> > >>>
> > >>> What is the lower level device of your DRBD resource? The
> > >>> whole md0,
> > >>> a
> > >>> partition on md0, a lv on a vg on a pv on md0?
> > >>>
> > >>> ... so if the lower level device has been resized on both
> > >>> nodes, DRBD
> > >>> should definitely grow on a "drbdadm resize".
> > >>>
> > >>> Did I mention that on starting DRBD it is resized
> > >>> automatically if a
> > >>> bigger lower level device is detected? ... have a look at
> > >>> the
> > >>> kernel
> > >>> logs...
> > >>>
> > >>> Regards,
> > >>> Andreas
> > >>>
> > >>> --
> > >>> Need help with DRBD?
> > >>> http://www.hastexo.com/now
> > >>>
> > >>> >
> > >>> > Gerald
> > >>> >
> > >>> >
> > >>> > ----- Original Message -----
> > >>> >> From: "Gerald Brandt" <gbr at majentis.com>
> > >>> >> To: drbd-user at lists.linbit.com
> > >>> >> Sent: Tuesday, October 18, 2011 7:08:55 AM
> > >>> >> Subject: Re: [DRBD-user] Increasing a DRBD array
> > >>> >>
> > >>> >> Hi,
> > >>> >>
> > >>> >> Okay, this is my list of what to do, and in what order:
> > >>> >>
> > >>> >> 1. remove the primary from DRBD
> > >>> >> 2. add the physical disk to the primary
> > >>> >> 3. add the primary back to DRBD and allow resync.
> > >>> >> 4. remove the secondary from DRBD
> > >>> >> 5. add the physical disk to the secondary
> > >>> >> 6. add the secondary back to DRBD and allow resync.
> > >>> >> 7. fdisk and add the disk to the RAID array on primary
> > >>> >> and
> > >>> >> secondary
> > >>> >> 8. grow the RAID array on the primary and secondary
> > >>> >> (while
> > >>> >> mounted
> > >>> >> or
> > >>> >> not mounted?)
> > >>> >> 9. drbdadm resize on the primary (can this be done
> > >>> >> while
> > >>> >> drbd is
> > >>> >> in
> > >>> >> use?)
> > >>> >> 10. expand the file system on the primary only (via
> > >>> >> Citrix
> > >>> >> XenServer
> > >>> >> (iSCSI))
> > >>> >>
> > >>> >> That sounds about right. As long as I don't lose data.
> > >>> >> (yeah,
> > >>> >> backups. That's left up to my client, but becomes my
> > >>> >> problem if
> > >>> >> they failed.)
> > >>> >>
> > >>> >> Gerald
> > >>> >>
> > >>> >>
> > >>> >>
> > >>> >> ----- Original Message -----
> > >>> >>> From: "Florian Haas" <florian at hastexo.com>
> > >>> >>> To: drbd-user at lists.linbit.com
> > >>> >>> Sent: Tuesday, October 18, 2011 3:53:11 AM
> > >>> >>> Subject: Re: [DRBD-user] Increasing a DRBD array
> > >>> >>>
> > >>> >>> On 2011-10-17 22:19, Matt Graham wrote:
> > >>> >>>>>> Your DRBD uses the MD RAID as its backing device,
> > >>> >>>>>> doesn't it?
> > >>> >>>>>> That means you'll have add the drive and resize
> > >>> >>>>>> your
> > >>> >>>>>> RAID
> > >>> >>>>>> first,
> > >>> >>>>>> then
> > >>> >>>>>> you can resize your DRBD, and then expand the file
> > >>> >>>>>> system.
> > >>> >>>>> Definitely built on top of the RAID. I'm a bit
> > >>> >>>>> confused
> > >>> >>>>> by
> > >>> >>>>> your
> > >>> >>>>> answer though. Step 8 grows the RAID array, and I
> > >>> >>>>> don't
> > >>> >>>>> have a
> > >>> >>>>> step
> > >>> >>>>> to grow DRBD. Does that mean I'm missing something?
> > >>> >>>>
> > >>> >>>> 8. mdadm --grow /dev/md0 (other mdadm options)
> > >>> >>>> Do that on both machines.
> > >>> >>>> 8.1 (on secondary) drbdsetup /dev/drbd0 resize
> > >>> >>>> 8.2 (on primary) drbdsetup /dev/drbd0 resize
> > >>> >>>
> > >>> >>> There's a "resize" subcommand for drbdadm too; no need
> > >>> >>> to
> > >>> >>> use
> > >>> >>> drbdsetup.
> > >>> >>> And it needs to be done on one node only. Just do
> > >>> >>> "drbdadm
> > >>> >>> resize
> > >>> >>> <resourcename>" on the primary while the two nodes are
> > >>> >>> connected,
> > >>> >>> and
> > >>> >>> you're good to go.
> > >>> >>>
> > >>> >>>> The softRAID is below the DRBD device. Once the
> > >>> >>>> softRAID is
> > >>> >>>> larger, the DRBD device needs to be informed of
> > >>> >>>> this
> > >>> >>>> fact.
> > >>> >>>> When making arrays/devices larger, you go
> > >>> >>>> bottom-up;
> > >>> >>>> you make
> > >>> >>>> the hardware changes first, then softRAID changes,
> > >>> >>>> then DRBD
> > >>> >>>> changes, then filesystem changes.
> > >>> >>>> 9. (on primary) resize2fs /dev/drbd0
> > >>> >>>
> > >>> >>> If it's ext3 and if it supports on-line resizing, of
> > >>> >>> course. But
> > >>> >>> that
> > >>> >>> should be obvious.
> > >>> >>>
> > >>> >>> Cheers,
> > >>> >>> Florian
> > >>> >>>
> > >>> >>> --
> > >>> >>> Need help with DRBD?
> > >>> >>> http://www.hastexo.com/knowledge/drbd
> > >>> >>> _______________________________________________
> > >>> >>> drbd-user mailing list
> > >>> >>> drbd-user at lists.linbit.com
> > >>> >>> http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>> >>>
> > >>> >> _______________________________________________
> > >>> >> drbd-user mailing list
> > >>> >> drbd-user at lists.linbit.com
> > >>> >> http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>> >>
> > >>> > _______________________________________________
> > >>> > drbd-user mailing list
> > >>> > drbd-user at lists.linbit.com
> > >>> > http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> drbd-user mailing list
> > >>> drbd-user at lists.linbit.com
> > >>> http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> _______________________________________________
> > >>> drbd-user mailing list
> > >>> drbd-user at lists.linbit.com
> > >>> http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> drbd-user mailing list
> > >> drbd-user at lists.linbit.com
> > >> http://lists.linbit.com/mailman/listinfo/drbd-user
> > >>
> >
> >
> > --
> > Need help with DRBD?
> > http://www.hastexo.com/now
> >
> >
> >
> > _______________________________________________
> > drbd-user mailing list
> > drbd-user at lists.linbit.com
> > http://lists.linbit.com/mailman/listinfo/drbd-user
> >
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>