Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
I am also setting up a device > 4TB on 8.0.6 and even though the initial steps went fine, the sync
produced tons of errors. From /var/log/messages:
Sep 16 13:26:22 boromir kernel: drbd0: size = 7450 GB (7812499983 KB)
Sep 16 13:26:22 boromir kernel: drbd0: writing of bitmap took 377 jiffies
Sep 16 13:26:22 boromir kernel: drbd0: 7450 GB marked out-of-sync by on disk bit-map.
Sep 16 13:26:22 boromir kernel: drbd0: 7812499984 KB now marked out-of-sync by on disk bit-map.
Sep 16 13:26:23 boromir kernel: drbd0: reading of bitmap took 713 jiffies
Sep 16 13:26:23 boromir kernel: drbd0: recounting of set bits took additional 225 jiffies
Sep 16 13:26:23 boromir kernel: drbd0: 7450 GB marked out-of-sync by on disk bit-map.
Later on tons of messages like:
Sep 17 01:21:54 boromir kernel: drbd0: drbd0_worker [4652]:drbd_md_sync_page_io(,262145s,WRITE)
out of range md access!
Sep 17 01:21:54 boromir kernel: drbd0: drbd0_worker [4652]:drbd_md_sync_page_io(,262146s,WRITE)
out of range md access!
..
..
..
Sep 17 10:27:07 boromir kernel: drbd0: drbd0_worker [4652]:drbd_md_sync_page_io(,476909s,WRITE) out
of range md access!
Sep 17 10:27:07 boromir kernel: drbd0: Resync done (total 74427 sec; paused 0 sec; 104968 K/sec)
Sep 17 10:27:07 boromir kernel: drbd0: conn( SyncSource -> Connected ) pdsk( Inconsistent -> UpToDate )
Sep 17 10:27:07 boromir kernel: drbd0: Writing meta data super block now.
[root at boromir ~]# cat /proc/drbd
version: 8.0.6 (api:86/proto:86)
SVN Revision: 3048 build by root at faramir.ibb.gatech.edu, 2007-09-17 12:52:27
0: cs:Connected st:Primary/Secondary ds:UpToDate/UpToDate C r---
ns:1492773386 nr:0 dw:122724347 dr:1370051009 al:118784 bm:476838 lo:0 pe:0 ua:0 ap:0
resync: used:0/31 hits:487804412 misses:476838 starving:0 dirty:0 changed:476838
act_log: used:0/257 hits:245329868 misses:163350 starving:0 dirty:44566 changed:118784
If drbd 0.8 does not support above 8TB (unless you get the plus version), shouldn't it stop or warn
the user somewhere about this?
I assume the error messages shown are because the original metadata size is 128MB and it is
exceeding it since the total disk size if over 4TB, right?
Diego
Desjardins, Kristian wrote:
> I am trying to setup drbd (8.0.4) on a large raid system, 9.745 TB and
> it does not see all of the disk. I wasn't sure of the state of drbd
> w.r.t. block devices larger than 4TB. It seems to work, almost.. Also,
> I am using a custom OpenVZ RHEL kernel 2.6.18-8.1.8.el5.028stab039.1 and
> I can't compile 8.0.6 (8.0.5 compiles but upon loading it says that it
> is 8.0.4), the error output is way below..
>
> Thanks.
>
>
>
> drbd.conf:
>
> global { usage-count yes; }
>
> resource ftp1 {
> protocol C;
> disk {
> on-io-error detach;
> }
> syncer {
> rate 80M;
> al-extents 257;
> }
> net {
> cram-hmac-alg sha1; shared-secret "";
> }
> on ccrsftp1 {
> device /dev/drbd0;
> disk /dev/sdb1;
> address 192.168.0.1:7789;
> flexible-meta-disk internal;
> }
> on ccrsftp2 {
> device /dev/drbd0;
> disk /dev/sdb1;
> address 192.168.0.2:7789;
> flexible-meta-disk internal;
> }
> }
>
> drbdadm create-md ftp1
>
> Valid meta-data already in place, recreate new?
> [need to type 'yes' to confirm] yes
>
> Creating meta data...
> initialising activity log
> NOT initialized bitmap (290420 KB)
> New drbd meta data block sucessfully created.
>
>
> drbdadm up ftp1
>
> output from kernel:
>
> drbd0: disk( Diskless -> Attaching )
> drbd0: No usable activity log found.
> drbd0: max_segment_size ( = BIO size ) = 32768
> drbd0: drbd_bm_resize called with capacity == 19032384136
> drbd0: resync bitmap: bits=2379048017 words=37172626
> drbd0: size = 9075 GB (9516192068 KB)
> drbd0: writing of bitmap took 1824 jiffies
> drbd0: 9075 GB marked out-of-sync by on disk bit-map.
> drbd0: 9516192068 KB now marked out-of-sync by on disk bit-map.
> drbd0: BUG! md_sync_timer expired! Worker calls drbd_md_sync().
> drbd0: reading of bitmap took 7888 jiffies
> drbd0: recounting of set bits took additional 205 jiffies
> drbd0: 9075 GB marked out-of-sync by on disk bit-map.
> drbd0: disk( Attaching -> Inconsistent )
> drbd0: conn( StandAlone -> Unconnected )
> drbd0: receiver (re)started
> drbd0: conn( Unconnected -> WFConnection )
>
>
> parted /dev/sdb
>
> GNU Parted 1.8.1
> Using /dev/sdb
> Welcome to GNU Parted! Type 'help' to view a list of commands.
> (parted) p
>
> Model: DELL PERC 5/E Adapter (scsi)
> Disk /dev/sdb: 9745GB
> Sector size (logical/physical): 512B/512B
> Partition Table: gpt
>
> Number Start End Size File system Name Flags
> 1 17.4kB 9745GB 9745GB xfs ftp1
>
> (parted)
>
>
> 8.0.6 compile error:
>
> drbd-8.0.6/drbd/drbd_main.c: In function 'is_valid_state_transition':
> drbd-8.0.6/drbd/drbd_main.c:599: error: 'SS_NoNetConfig' undeclared
> (first use in this function)
> drbd-8.0.6/drbd/drbd_main.c:599: error: (Each undeclared identifier is
> reported only once
> drbd-8.0.6/drbd/drbd_main.c:599: error: for each function it appears
> in.)
>
> 8.0.5 saying it is 8.0.4:
>
> [drbd-8.0.5]$ strings drbd/drbd.ko | grep 8.0.4
> version: 8.0.4 (api:%d/proto:%d)
> <6>drbd: initialised. Version: 8.0.4 (api:%d/proto:%d)
> description=drbd - Distributed Replicated Block Device v8.0.4
> _______________________________________________
> drbd-user mailing list
> drbd-user at lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user