Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On 10/22/2011 05:47 PM, Gerald Brandt wrote:
> Okay, this is the plan for changing an primary/secondary drbd from 'meta-disk /dev/sda6[1] to 'flexible-meta-disk /dev/sda6'
>
> sda6 is already 512 MB in size, which in theory will let me go to 16 TB storage (128 MB is 4 TB storage).
yes, 512MB is fine
>
> on secondary:
> 1. drbdadm down iscsi.target.0
> 2. edit drbd.conf to reflect meta-disk change (also on primary?)
no, do it on the secondary only for now
> 3. drbdadm create-md iscsi.target.0
> 4. drbdadm up iscsi.target.0
> 5. wait for full sync to take place
>
> on primary
you mean on the previous primary ... as you might want to do a
switchover to the already prepared secondary ....
> 6-10 same as 1 - 5 above
>
> At with point I should be able to do drbdadm resize iscsi.target.0, and I should see 6TB of storage.
after point 4 where the previous primary connects to the already updated
previous secondary the resize starts automatically as the two nodes
detect they both have more diskspace and metadata that can handle this
... so you will see a resync of the "missing" 2TB
>
> That that sound right to everyone? It should give me no downtime and the ability to have more than 4TB storage.
No downtime? ... except the time you need to switch Roles and therefore
migrate services ... but I assume you let your cluster manager do this job.
Regards,
Andreas
--
Need help with DRBD?
http://www.hastexo.com/now
>
>
> Gerald
>
> ----- Original Message -----
>> From: "Gerald Brandt" <gbr at majentis.com>
>> To: "Andreas Kurz" <andreas at hastexo.com>
>> Cc: "drbd-user" <drbd-user at lists.linbit.com>
>> Sent: Saturday, October 22, 2011 8:33:15 AM
>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>
>> Hi,
>>
>>
>>
>> ----- Original Message -----
>>> From: "Andreas Kurz" <andreas at hastexo.com>
>>> To: "drbd-user" <drbd-user at lists.linbit.com>
>>> Sent: Friday, October 21, 2011 6:12:11 PM
>>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>
>>> On 10/21/2011 11:39 PM, Gerald Brandt wrote:
>>>> Hi,
>>>>
>>>> I just saw that (google is my friend). Can I change that on a
>>>> running drbd system?
>>>
>>> hmm ... never tried changing meta-date that way ... shutdown,
>>> dump-md,
>>> reconfigure, create-md, restore-md might work ... maybe Lars has a
>>> hint ...
>>>
>>> I would bring DRBD down on both nodes, stop it when all is in sync
>>> and
>>> recreate the meta data after changing the config and then skip the
>>> initial sync when bringing them up.
>>>
>>
>> I really can't bring the nodes down. I can bring down one at a time,
>> but the systems have to stay running.
>>
>>>>
>>>> ie:
>>>>
>>>> original:
>>>>
>>>> on iscsi-filer-1 {simply use "meta-disk internal;"
>>>> device /dev/drbd1;
>>>> disk /dev/md0;
>>>> address 192.168.95.1:7789;
>>>> meta-disk /dev/sda6[1];
>>>> }
>>>>
>>>> on iscsi-filer-2 {
>>>> device /dev/drbd1;
>>>> disk /dev/md0;
>>>> address 192.168.95.2:7789;
>>>> meta-disk /dev/sda6[1];
>>>> }
>>>>
>>>> new:
>>>>
>>>> on iscsi-filer-1 {
>>>> device /dev/drbd1;
>>>> disk /dev/md0;
>>>> address 192.168.95.1:7789;
>>>> flexible-meta-disk /dev/sda6[1];
>>>
>>> no ... that index thing only works for "static" meta-disk ...
>>> remove
>>> the
>>> [1] and resize /dev/sda6 if its not bigger than 196MB.
>>
>> I'm not sure I understand. /dev/sda6 is already 512 MB (I think).
>> Should I change to:
>>
>> on iscsi-filer-1 {
>> device /dev/drbd1;
>> disk /dev/md0;
>> address 192.168.95.1:7789;
>> flexible-meta-disk /dev/sda6;
>> }
>>
>> on iscsi-filer-2 {
>> device /dev/drbd1;
>> disk /dev/md0;
>> address 192.168.95.2:7789;
>> flexible-meta-disk /dev/sda6;
>> }
>>
>> or would this be better:
>>
>> on iscsi-filer-1 {
>> device /dev/drbd1;
>> disk /dev/md0;
>> address 192.168.95.1:7789;
>> meta-disk internal;
>> }
>>
>> on iscsi-filer-2 {
>> device /dev/drbd1;
>> disk /dev/md0;
>> address 192.168.95.2:7789;
>> meta-disk internal;
>> }
>>
>>
>> I'll go back to my lists to see if I'm doing things right.
>>
>> 1. bring down the secondary
>> 2. change the secondary to 'flexible-meta-data /dev/sda6' in
>> drbd.conf on primary and secondary.
>> 3. bring secondary back up (may re-sync entire disk, not a serious
>> issue, just time)
>> 4. repeat process for primary after re-sync (may cause another
>> complete resync).
>>
>> Gerald
>>
>>
>>>
>>> Regards,
>>> Andreas
>>>
>>> --
>>> Need help with DRBD?
>>> http://www.hastexo.com/now
>>>
>>>> }
>>>>
>>>> on iscsi-filer-2 {
>>>> device /dev/drbd1;
>>>> disk /dev/md0;
>>>> address 192.168.95.2:7789;
>>>> flexible-meta-disk /dev/sda6[1];
>>>> }
>>>>
>>>> Then reboot primary, followed by reboot secondary (after sync),
>>>> and
>>>> all will be well?
>>>>
>>>> Sorry if these seem to be noob questions. I just want to be 100%
>>>> sure, as the file servers have live data on them.
>>>>
>>>> Gerald
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Andreas Kurz" <andreas at hastexo.com>
>>>>> To: drbd-user at lists.linbit.com
>>>>> Sent: Friday, October 21, 2011 4:26:13 PM
>>>>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>>>
>>>>> On 10/21/2011 10:48 PM, Gerald Brandt wrote:
>>>>>> Hi,
>>>>>>
>>>>>> DRBD is running directly on md0. /dev/drbd1 is then exported
>>>>>> via
>>>>>> iSCSI.
>>>>>>
>>>>>> The logs show:
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332010] drbd:
>>>>>> initialized.
>>>>>> Version: 8.3.7 (api:88/proto:86-91)
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332012] drbd:
>>>>>> GIT-hash:
>>>>>> ea9e28dbff98e331a62bcbcc63a6135808fe2917 build by root at filer-1,
>>>>>> 2011-03-05 08:29:38
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332014] drbd:
>>>>>> registered as
>>>>>> block device major 147
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.332015] drbd:
>>>>>> minor_table @
>>>>>> 0xffff88021dbaf300
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334144] block
>>>>>> drbd1:
>>>>>> Starting worker thread (from cqueue [1489])
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334177] block
>>>>>> drbd1:
>>>>>> ==>
>>>>>> truncating very big lower level device to currently maximum
>>>>>> possible
>>>>>> 8587575296 sectors <==
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334179] block
>>>>>> drbd1:
>>>>>> ==>>
>>>>>> using internal or flexible meta data may help <<==
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.334188] block
>>>>>> drbd1:
>>>>>> disk(
>>>>>> Diskless -> Attaching )
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.353306] Loading
>>>>>> iSCSI
>>>>>> transport class v2.0-870.
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.360582] skge eth0:
>>>>>> disabling interface
>>>>>> Oct 21 15:34:53 iscsi-filer-1 iscsid: iSCSI logger with
>>>>>> pid=1515
>>>>>> started!
>>>>>> Oct 21 15:34:53 iscsi-filer-1 kernel: [ 7.381956] iscsi:
>>>>>> registered
>>>>>> transport (iser)
>>>>>> Oct 21 15:34:53 iscsi-filer-1 init: ssh main process (1162)
>>>>>> terminated
>>>>>> with status 255
>>>>>> Oct 21 15:34:53 iscsi-filer-1 postfix/master[1411]: reload --
>>>>>> version
>>>>>> 2.7.0, configuration /etc/postfix
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584678] block
>>>>>> drbd1:
>>>>>> Found
>>>>>> 57 transactions (3507 active extents) in activity log.
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584684] block
>>>>>> drbd1:
>>>>>> Method
>>>>>> to ensure write ordering: barrier
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584691] block
>>>>>> drbd1:
>>>>>> Backing device's merge_bvec_fn() = ffffffffa00c0100
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584694] block
>>>>>> drbd1:
>>>>>> max_segment_size ( = BIO size ) = 4096
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584698] block
>>>>>> drbd1:
>>>>>> Adjusting my ra_pages to backing device's (32 -> 96)
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.584704] block
>>>>>> drbd1:
>>>>>> drbd_bm_resize called with capacity == 8587575296
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.603470] block
>>>>>> drbd1:
>>>>>> resync
>>>>>> bitmap: bits=1073446912 words=16772608
>>>>>> Oct 21 15:34:54 iscsi-filer-1 kernel: [ 7.603474] block
>>>>>> drbd1:
>>>>>> size =
>>>>>> 4095 GB (4293787648 KB)
>>>>>>
>>>>>> The lines in yellow bug me. I don't recall see them before.
>>>>>>
>>>>>> I had a 4 disk RAID-6 md0 (4x2TB = 4 TB RAID-6). I added a
>>>>>> single
>>>>>> drive
>>>>>> (5x2TB = 6 TB array).
>>>>>
>>>>> Not using "meta-disk internal" or "flexible-meta-disk" limits
>>>>> the
>>>>> device
>>>>> size to 4TB (=128MB meta data size) ... change your metadata
>>>>> config
>>>>> ...
>>>>> as the logs suggest ... if you want to use all 6TB
>>>>>
>>>>> Regards,
>>>>> Andreas
>>>>>
>>>>> --
>>>>> Need help with DRBD?
>>>>> http://www.hastexo.com/now
>>>>>
>>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>> Gerald
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>> From: "Andreas Kurz" <andreas at hastexo.com>
>>>>>> To: drbd-user at lists.linbit.com
>>>>>> Sent: Friday, October 21, 2011 2:55:06 PM
>>>>>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>>>>
>>>>>> On 10/21/2011 09:30 PM, Gerald Brandt wrote:
>>>>>> > Hi,
>>>>>> >
>>>>>> > I've successfully resize the lower level RAID-6 array,
>>>>>> > and
>>>>>> > grown
>>>>>> > it. I'm now attempting to resize drbd, and nothing
>>>>>> > seems
>>>>>> > to
>>>>>> > happen.
>>>>>> >
>>>>>> > /dev/md0 is definitely bigger.
>>>>>> >
>>>>>> > What should I see during a drbd resize?
>>>>>>
>>>>>> You should see a DRBD resync of the newly added space.
>>>>>>
>>>>>> What is the lower level device of your DRBD resource? The
>>>>>> whole md0,
>>>>>> a
>>>>>> partition on md0, a lv on a vg on a pv on md0?
>>>>>>
>>>>>> ... so if the lower level device has been resized on both
>>>>>> nodes, DRBD
>>>>>> should definitely grow on a "drbdadm resize".
>>>>>>
>>>>>> Did I mention that on starting DRBD it is resized
>>>>>> automatically if a
>>>>>> bigger lower level device is detected? ... have a look at
>>>>>> the
>>>>>> kernel
>>>>>> logs...
>>>>>>
>>>>>> Regards,
>>>>>> Andreas
>>>>>>
>>>>>> --
>>>>>> Need help with DRBD?
>>>>>> http://www.hastexo.com/now
>>>>>>
>>>>>> >
>>>>>> > Gerald
>>>>>> >
>>>>>> >
>>>>>> > ----- Original Message -----
>>>>>> >> From: "Gerald Brandt" <gbr at majentis.com>
>>>>>> >> To: drbd-user at lists.linbit.com
>>>>>> >> Sent: Tuesday, October 18, 2011 7:08:55 AM
>>>>>> >> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>>>> >>
>>>>>> >> Hi,
>>>>>> >>
>>>>>> >> Okay, this is my list of what to do, and in what order:
>>>>>> >>
>>>>>> >> 1. remove the primary from DRBD
>>>>>> >> 2. add the physical disk to the primary
>>>>>> >> 3. add the primary back to DRBD and allow resync.
>>>>>> >> 4. remove the secondary from DRBD
>>>>>> >> 5. add the physical disk to the secondary
>>>>>> >> 6. add the secondary back to DRBD and allow resync.
>>>>>> >> 7. fdisk and add the disk to the RAID array on primary
>>>>>> >> and
>>>>>> >> secondary
>>>>>> >> 8. grow the RAID array on the primary and secondary
>>>>>> >> (while
>>>>>> >> mounted
>>>>>> >> or
>>>>>> >> not mounted?)
>>>>>> >> 9. drbdadm resize on the primary (can this be done
>>>>>> >> while
>>>>>> >> drbd is
>>>>>> >> in
>>>>>> >> use?)
>>>>>> >> 10. expand the file system on the primary only (via
>>>>>> >> Citrix
>>>>>> >> XenServer
>>>>>> >> (iSCSI))
>>>>>> >>
>>>>>> >> That sounds about right. As long as I don't lose data.
>>>>>> >> (yeah,
>>>>>> >> backups. That's left up to my client, but becomes my
>>>>>> >> problem if
>>>>>> >> they failed.)
>>>>>> >>
>>>>>> >> Gerald
>>>>>> >>
>>>>>> >>
>>>>>> >>
>>>>>> >> ----- Original Message -----
>>>>>> >>> From: "Florian Haas" <florian at hastexo.com>
>>>>>> >>> To: drbd-user at lists.linbit.com
>>>>>> >>> Sent: Tuesday, October 18, 2011 3:53:11 AM
>>>>>> >>> Subject: Re: [DRBD-user] Increasing a DRBD array
>>>>>> >>>
>>>>>> >>> On 2011-10-17 22:19, Matt Graham wrote:
>>>>>> >>>>>> Your DRBD uses the MD RAID as its backing device,
>>>>>> >>>>>> doesn't it?
>>>>>> >>>>>> That means you'll have add the drive and resize
>>>>>> >>>>>> your
>>>>>> >>>>>> RAID
>>>>>> >>>>>> first,
>>>>>> >>>>>> then
>>>>>> >>>>>> you can resize your DRBD, and then expand the file
>>>>>> >>>>>> system.
>>>>>> >>>>> Definitely built on top of the RAID. I'm a bit
>>>>>> >>>>> confused
>>>>>> >>>>> by
>>>>>> >>>>> your
>>>>>> >>>>> answer though. Step 8 grows the RAID array, and I
>>>>>> >>>>> don't
>>>>>> >>>>> have a
>>>>>> >>>>> step
>>>>>> >>>>> to grow DRBD. Does that mean I'm missing something?
>>>>>> >>>>
>>>>>> >>>> 8. mdadm --grow /dev/md0 (other mdadm options)
>>>>>> >>>> Do that on both machines.
>>>>>> >>>> 8.1 (on secondary) drbdsetup /dev/drbd0 resize
>>>>>> >>>> 8.2 (on primary) drbdsetup /dev/drbd0 resize
>>>>>> >>>
>>>>>> >>> There's a "resize" subcommand for drbdadm too; no need
>>>>>> >>> to
>>>>>> >>> use
>>>>>> >>> drbdsetup.
>>>>>> >>> And it needs to be done on one node only. Just do
>>>>>> >>> "drbdadm
>>>>>> >>> resize
>>>>>> >>> <resourcename>" on the primary while the two nodes are
>>>>>> >>> connected,
>>>>>> >>> and
>>>>>> >>> you're good to go.
>>>>>> >>>
>>>>>> >>>> The softRAID is below the DRBD device. Once the
>>>>>> >>>> softRAID is
>>>>>> >>>> larger, the DRBD device needs to be informed of
>>>>>> >>>> this
>>>>>> >>>> fact.
>>>>>> >>>> When making arrays/devices larger, you go
>>>>>> >>>> bottom-up;
>>>>>> >>>> you make
>>>>>> >>>> the hardware changes first, then softRAID changes,
>>>>>> >>>> then DRBD
>>>>>> >>>> changes, then filesystem changes.
>>>>>> >>>> 9. (on primary) resize2fs /dev/drbd0
>>>>>> >>>
>>>>>> >>> If it's ext3 and if it supports on-line resizing, of
>>>>>> >>> course. But
>>>>>> >>> that
>>>>>> >>> should be obvious.
>>>>>> >>>
>>>>>> >>> Cheers,
>>>>>> >>> Florian
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> Need help with DRBD?
>>>>>> >>> http://www.hastexo.com/knowledge/drbd
>>>>>> >>> _______________________________________________
>>>>>> >>> drbd-user mailing list
>>>>>> >>> drbd-user at lists.linbit.com
>>>>>> >>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>> >>>
>>>>>> >> _______________________________________________
>>>>>> >> drbd-user mailing list
>>>>>> >> drbd-user at lists.linbit.com
>>>>>> >> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>> >>
>>>>>> > _______________________________________________
>>>>>> > drbd-user mailing list
>>>>>> > drbd-user at lists.linbit.com
>>>>>> > http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> drbd-user mailing list
>>>>>> drbd-user at lists.linbit.com
>>>>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> drbd-user mailing list
>>>>>> drbd-user at lists.linbit.com
>>>>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> drbd-user mailing list
>>>>> drbd-user at lists.linbit.com
>>>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>>>
>>>
>>>
>>> --
>>> Need help with DRBD?
>>> http://www.hastexo.com/now
>>>
>>>
>>>
>>> _______________________________________________
>>> drbd-user mailing list
>>> drbd-user at lists.linbit.com
>>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>>
>> _______________________________________________
>> drbd-user mailing list
>> drbd-user at lists.linbit.com
>> http://lists.linbit.com/mailman/listinfo/drbd-user
>>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 286 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20111024/00736810/attachment.pgp>