[DRBD-user] Expanding a cluster

Justin Edmands shockwavecs at gmail.com
Thu Jan 31 18:04:36 CET 2013

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


I'm on the fence about the amount of time it will take to degrade and
rebuild a RAID6 at 16 drives (x2 systems).

Anyone against the idea of:
Backup data friday night through saturday morning
stop drbd and heartbeat on node2
replace all drives on node2
build raid 6 and match setup/sizes from node1
initialize metadata, etc.
start drbd and heartbeat
let it sync
make node2 primary
repeat steps for node1

How does this sound? You'll be risking the data only during syncing process
but you would be doing that for 12-14 drives (per system) to tell them to
rebuild the RAID. Either way you have tape or some form of backup...DRBD is
not a backup :) There I said it.

Let's start off with, you're risking your data during this process no
matter what way you go about it. I had Seagate ES drives die during a
rebuild of node2 because the syncing 100% reading from node1 and 100%
writing to node2 was just too much for the older drives to handle. I think
the mixing of drives works in theory but it something goes awry in the
process you'll do your best to get it running and most likely stop the
upgrade process temporarily. This will leave you with some 1TB and some 2TB
disks in a running state with fear of further breaking it,etc.




On Thu, Jan 31, 2013 at 11:20 AM, Adam Goryachev <
mailinglists at websitemanagers.com.au> wrote:

>  On 01/02/13 02:58, Marcelo Pereira wrote:
>
> Hello Everyone,
>
>  I'm about to perform an upgrade on my servers and I was wondering how to
> do that.
>
>  Here is the scenario:
>
>  Server A has 16x 1Tb hard drives, under RAID-6.
> Server B has 16x 1Tb hard drives, under RAID-6.
>
>  And both are in sync, using DRBD.
>
>  I though about replacing the hard drives for 2Tb units, one by one.
>
>  So, on each run, I would:
>
>    - Remove a 1Tb disk
>    - Add a 2Tb disk
>    - Wait for it to rebuild the RAID
>
> After replacing ALL disks, I would expand the RAID unit, on each server.
>
>  However, I was wondering how DRBD would "like" this procedure.
>
>  I know that, before "expanding" the RAID, the cluster size, and the
> block numbers would remain the same, as I would be "wasting" the extra
> space on the newly added drives.
>
>  So, after "both" servers have all the drives replaces, and the RAID is
> properly rebuild. Would that be a problem to expand it? How would DRBD
> handle it?
>
>  I will appreciate any comment or suggestion here.
>
> DRBD will work perfectly...
>
> You probably need to do the following:
> 1) Pull one drive and replace (you could do one on each server at the same
> time, although better/safer to do one server at a time)
> 2) Wait for rebuild to complete
> 3) Repeat for all disks on BOTH servers
> 4) Resize the RAID array on each server
> 5) Resize DRBD (see the fantastic online manual for your version of DRBD
> for the details)
> 6) Resize the underlying filesystem or whatever
>
> BTW, depending on your kernel version, and/or RAID (I'm assuming linux
> software raid), you might like to query the linux-raid list to see if you
> can ADD the new drive, tell md that this new drive is replacing drive X,
> this way you avoid degrading the RAID array, hence lose less performance
> during the rebuild, and have a lower risk of disk failure and especially
> URE (Unrecoverable Read Error) during the rebuilds.
>
> Regards,
> Adam
>
>
> --
> Adam Goryachev
> Website Managerswww.websitemanagers.com.au
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20130131/87dad3bf/attachment.htm>


More information about the drbd-user mailing list