[DRBD-user] Linstor Cluster Limits

[Robert Altnoeder]

We have not reached the practical limit so far, and the theoretical
limits are not practically realistic. The theoretical absolute limit on
the number of nodes and the number of resources is somewhat more than 2
billion each. For DRBD resources, the maximum number of resources is
limited by the size of the TCP/IP port number pool, since each DRBD
resource is assigned a unique TCP/IP port number. Assuming that each
resource has at least one volume, the maximum number of resources of
each type is normally somewhat more than 1 million per resource type,
due to operating system limitations on the number of devices per device
driver type. For DRBD, the maximum number of volumes per resource is
65534, and the maximum number of volumes for all resources is ~ 1 million.

We have some partners who work on fairly large clusters, but I do not
have exact numbers, so I would have to ask what the exact number of
nodes and resources is.

In our own test environment, we have run clusters with about 50 nodes,
and we have run configurations with more than 2000 resource definitions
and more than 4500 resources per cluster (all of those were replicated
DRBD resources, not just local volumes). This was close to the practical
limit of network traffic between DRBD resources on a single network
interface (so that replication connection may become unreliable due to
network traffic overload), but it works normally if that traffic is
distributed using multiple network interfaces (or alternatively, a
single faster one, of course).

Anyhow, knowing the exact limits in LINSTOR only rules out LINSTOR's
software logic as the source of a problem, but since most of the
external environment that LINSTOR runs in - operating system, device
drivers, utilities, filesystems, etc. - was not designed with
determinism in mind. There is simply no culture of knowing the exact
limitations of all the components and making sure that their
implementations correctly and consistently implement them. As a result,
the actual limits of the system as a whole are unknown.

br,
Robert


On 6/17/19 4:48 PM, kvaps wrote:
> Hello,
>
> Is there any known limits for the single linstor cluster?
> How much nodes and resources can be created within the same cluster?
> What is largest deployment do you have in your experience?
>
> Thank in advance!
>
> - kvaps
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user


-- 
Robert ALTNOEDER - Software Developer
+43-1-817-82-92 x72 <tel:+4318178292>
robert.altnoeder at linbit.com <mailto:robert.altnoeder at linbit.com>

LIN <http://www.linbit.com/en/>BIT <http://www.linbit.com/en/> | Keeping
the Digital World Running
DRBD HA - Disaster Recovery - Software-defined Storage
t <https://twitter.com/linbit> / f
<https://www.facebook.com/pg/linbitdrbd/posts/> / in
<https://www.linkedin.com/company/linbit> / y
<https://www.youtube.com/user/linbit> / g+
<https://plus.google.com/+Linbit/about>

DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.