[DRBD-user] Linstor Cluster Limits

[kvaps]

Robert,

Thanks a lot for the detailed answer!

- kvaps

On Mon, Jun 17, 2019 at 7:25 PM Robert Altnoeder
<robert.altnoeder at linbit.com> wrote:
>
> We have not reached the practical limit so far, and the theoretical
> limits are not practically realistic. The theoretical absolute limit on
> the number of nodes and the number of resources is somewhat more than 2
> billion each. For DRBD resources, the maximum number of resources is
> limited by the size of the TCP/IP port number pool, since each DRBD
> resource is assigned a unique TCP/IP port number. Assuming that each
> resource has at least one volume, the maximum number of resources of
> each type is normally somewhat more than 1 million per resource type,
> due to operating system limitations on the number of devices per device
> driver type. For DRBD, the maximum number of volumes per resource is
> 65534, and the maximum number of volumes for all resources is ~ 1 million.
>
> We have some partners who work on fairly large clusters, but I do not
> have exact numbers, so I would have to ask what the exact number of
> nodes and resources is.
>
> In our own test environment, we have run clusters with about 50 nodes,
> and we have run configurations with more than 2000 resource definitions
> and more than 4500 resources per cluster (all of those were replicated
> DRBD resources, not just local volumes). This was close to the practical
> limit of network traffic between DRBD resources on a single network
> interface (so that replication connection may become unreliable due to
> network traffic overload), but it works normally if that traffic is
> distributed using multiple network interfaces (or alternatively, a
> single faster one, of course).
>
> Anyhow, knowing the exact limits in LINSTOR only rules out LINSTOR's
> software logic as the source of a problem, but since most of the
> external environment that LINSTOR runs in - operating system, device
> drivers, utilities, filesystems, etc. - was not designed with
> determinism in mind. There is simply no culture of knowing the exact
> limitations of all the components and making sure that their
> implementations correctly and consistently implement them. As a result,
> the actual limits of the system as a whole are unknown.
>
> br,
> Robert
>
>
> On 6/17/19 4:48 PM, kvaps wrote:
> > Hello,
> >
> > Is there any known limits for the single linstor cluster?
> > How much nodes and resources can be created within the same cluster?
> > What is largest deployment do you have in your experience?
> >
> > Thank in advance!
> >
> > - kvaps
> > _______________________________________________
> > drbd-user mailing list
> > drbd-user at lists.linbit.com
> > http://lists.linbit.com/mailman/listinfo/drbd-user
>
>
> --
> Robert ALTNOEDER - Software Developer
> +43-1-817-82-92 x72 <tel:+4318178292>
> robert.altnoeder at linbit.com <mailto:robert.altnoeder at linbit.com>
>
> LIN <http://www.linbit.com/en/>BIT <http://www.linbit.com/en/> | Keeping
> the Digital World Running
> DRBD HA - Disaster Recovery - Software-defined Storage
> t <https://twitter.com/linbit> / f
> <https://www.facebook.com/pg/linbitdrbd/posts/> / in
> <https://www.linkedin.com/company/linbit> / y
> <https://www.youtube.com/user/linbit> / g+
> <https://plus.google.com/+Linbit/about>
>
> DRBD® and LINBIT® are registered trademarks of LINBIT, Austria.
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user