Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
/ 2004-08-30 16:57:54 -0400 \ Ian Samuel: > > Hello, > > I tried today to set up DRBD over a software-striped disk set (2 x 1.25 TB) > for a total DRBD drive of 2.5TB. > > Unfortunatley, although the source code for v0.7.2 mentions a maximum size of > 3.8TB - the size reported back by DRBD as only 310 GB instead. > > I can mount and use the /dev/md5 device with reiserfs (I have enabled >2TB > filesystem support in the kernel), it is only when I try to layer DRBD over > top that I have a problem with this size of device. > > I am using another 50GB drbd parition on the same systems and that works fine, > and using only 1 disk (1.25GB) also works fine. > > Kernel is 2.6.8.1, with DRBD 0.7.2 on Debian sarge. > Unfortunately I have to admit that we have several pieces in the code were we still use "unsigned long" as size, sometimes in sectors and sometimes in kilo byte. So the mentioned 3.8 TB are a limit by choice of maximum bitmap size. the actual limit due to code leftovers from previous times seems to be much lower. My current tests (and a quick glance at the code) suggest that you can use up to 1.8TB safely, probably just below 2TB, seemingly even just below 2.2TB, if meta data is internal (not recommended at that size because of seektime for meta data updates) and thus the _effective_ (user visible) size is back at < 2TB ... FYI: ,--- just seen in my test setup, working: |drbd0: Creating state block |drbd0: resync bitmap: bits=503316736 words=15728648 |drbd0: size = 2013266944 KB |drbd0: Assuming that all blocks are out of sync (aka FullSync) |drbd0: 2013266944 KB now marked out-of-sync by on disk bit-map. `--- this is 0x78000400 kb. added an other 0x0800000 kb, which would end up as 2TB + 1MB, and it just blew up :-( Sorry. We should at least refuse to setup devices of that size until we sort out these issues. I can even imaging weird OOM conditions or oopses related to too large devices (if you happen to hit some special number of sectors ...) > Is it unrealistic to use a single DRBD device this large? I am hoping it is > a simple 32-bit int overflow. No, unfortunately it is not that simple. Several unsigned longs need to be converted to sector_t, some of them to loff_t; several calculations and far dependencies have to be reviewed, and mached up again. General code cleanup of leftovers of former times has to follow, and maybe we need to forcefully break all 2.4. compatibility. We'll think it over and let you know. Meanwhile, please use devices of less than 2TB to be on the safe side. Lars Ellenberg