Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
On 23/12/13 08:21 AM, Vladislav Bogdanov wrote:
> 23.12.2013 05:39, Digimer wrote:
>> On 22/12/13 09:28 PM, Digimer wrote:
>>> On 22/12/13 06:05 AM, Patrick Lists wrote:
>>>> Hi Digimer,
>>>>
>>>> On 12/22/2013 07:46 AM, Digimer wrote:
>>>>> Hi all,
>>>>>
>>>>> I am testing RHEL 7 and DRBD. I grabbed drbd from the Fedora 19
>>>>> repos
>>>>> as EPEL isn't up yet. It installed fine, but I can't seem to 'modprobe
>>>>> drbd'.
>>>>>
>>>>> Has anyone tried to get DRBD running on RHEL 7 yet? Mind sharing the
>>>>> secret sauce, if so? If not, I'll try installing from source.
>>>>
>>>> Have you checked the logs? Watch the output of 'journalctl -f' when you
>>>> modprobe drbd and it should tell you what's going on.
>>>>
>>>> Instead of installing from source I would just grab the source rpm,
>>>> check the spec file for any obvious incompatibilities with EL7, fix any
>>>> and rebuild it.
>>>>
>>>> Regards,
>>>> Patrick
>>>
>>> Ok, I realized that '--without-km' was set in the spec. I changed that
>>> to '--with-km' and tried building it again, but it failed this time. I'm
>>> not much of a programmer, so I'm hoping you (or others) might be able to
>>> help here.
>>>
>>> Here is the error, so far as I can tell:
>>>
>>> ====
>>> gcc -O2 -g -Wall -I. -I../drbd -I../drbd/compat -c -o drbdadm_parser.o
>>> drbdadm_parser.c
>>> UPD /root/rpmbuild/BUILD/drbd-8.4.3/drbd/compat.h
>>> CC [M] /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_buildtag.o
>>> CC [M] /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_bitmap.o
>>> gcc -O2 -g -Wall -I. -I../drbd -I../drbd/compat -c -o drbdadm_main.o
>>> drbdadm_main.c
>>> In file included from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_bitmap.c:35:0:
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_int.h:2515:0: warning:
>>> "idr_for_each_entry" redefined [enabled by default]
>>> #define idr_for_each_entry(idp, entry, id) \
>>> ^
>>> In file included from include/linux/cgroup.h:19:0,
>>> from include/net/netprio_cgroup.h:16,
>>> from include/linux/netdevice.h:48,
>>> from include/net/sock.h:51,
>>> from include/linux/connector.h:30,
>>> from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/linux/drbd.h:28,
>>> from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_bitmap.c:28:
>>> include/linux/idr.h:132:0: note: this is the location of the previous
>>> definition
>>> #define idr_for_each_entry(idp, entry, id) \
>>> ^
>>> CC [M] /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.o
>>> gcc -O2 -g -Wall -I. -I../drbd -I../drbd/compat -c -o drbdadm_adjust.o
>>> drbdadm_adjust.c
>>> gcc -O2 -g -Wall -I. -I../drbd -I../drbd/compat -c -o
>>> drbdtool_common.o drbdtool_common.c
>>> In file included from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.c:34:0:
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_int.h:2515:0: warning:
>>> "idr_for_each_entry" redefined [enabled by default]
>>> #define idr_for_each_entry(idp, entry, id) \
>>> ^
>>> In file included from include/linux/cgroup.h:19:0,
>>> from include/net/netprio_cgroup.h:16,
>>> from include/linux/netdevice.h:48,
>>> from include/net/sock.h:51,
>>> from include/linux/connector.h:30,
>>> from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/linux/drbd.h:28,
>>> from
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.c:33:
>>> include/linux/idr.h:132:0: note: this is the location of the previous
>>> definition
>>> #define idr_for_each_entry(idp, entry, id) \
>>> ^
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.c: In function
>>> 'drbd_proc_open':
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.c:320:3: error: implicit
>>> declaration of function 'PDE' [-Werror=implicit-function-declaration]
>>> return single_open(file, drbd_seq_show, PDE(inode)->data);
>>> ^
>>> /root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.c:320:53: error: invalid
>>> type argument of '->' (have 'int')
>>> return single_open(file, drbd_seq_show, PDE(inode)->data);
>>> ^
>>> cc1: some warnings being treated as errors
>>> make[3]: *** [/root/rpmbuild/BUILD/drbd-8.4.3/drbd/drbd_proc.o] Error 1
>>> make[2]: *** [_module_/root/rpmbuild/BUILD/drbd-8.4.3/drbd] Error 2
>>> make[2]: Leaving directory `/usr/src/kernels/3.10.0-54.0.1.el7.x86_64'
>>> make[1]: *** [kbuild] Error 2
>>> make[1]: Leaving directory `/root/rpmbuild/BUILD/drbd-8.4.3/drbd'
>>> make: *** [module] Error 2
>>> ====
>>>
>>> I'll keep poking at it and report back if I make progress... As of now
>>> though, I'm stuck. :)
>>
>> Building from 8.4.3 source errors out with the same problem.
>
> This one seems to be fixed (moved to drbd_wrappers.h and ifdef'ed) in 8.4.4.
I've got DRBD 8.4.4 installed and sync'ing on RHEL 7 beta! \o/
Here was the magic (based on a minimal install):
====
yum install -y gcc flex rpm-build wget kernel-devel
wget -c http://oss.linbit.com/drbd/8.4/drbd-8.4.4.tar.gz
tar -xvzf drbd-8.4.4.tar.gz
cd drbd-8.4.4
./configure \
--prefix=/usr \
--localstatedir=/var \
--sysconfdir=/etc \
--with-km \
--with-udev \
--with-pacemaker \
--with-bashcompletion \
--with-utils \
--without-xen \
--without-heartbeat
make
make install
modprobe drbd
====
Stuff that may or may not be relevant to someone later:
systemctl disable drbd.service
cat /etc/drbd.d/global_common.conf
====
# These are options to set for the DRBD daemon sets the default values for
# resources.
global {
# This tells DRBD that you allow it to report this installation to
# LINBIT for statistical purposes. If you have privacy concerns, set
# this to 'no'. The default is 'ask' which will prompt you each time
# DRBD is updated. Set to 'yes' to allow it without being prompted.
usage-count no;
# minor-count dialog-refresh disable-ip-verification
}
common {
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh;
/usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ;
reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh;
/usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger
; halt -f";
# split-brain "/usr/lib/drbd/notify-split-brain.sh root";
# out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";
# before-resync-target "/usr/lib/drbd/snapshot-resync-target-lvm.sh -p
15 -- -c 16k";
# after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;
# Hook into Pacemaker's fencing.
fence-peer "/usr/lib/drbd/crm-fence-peer.sh";
}
startup {
# wfc-timeout degr-wfc-timeout outdated-wfc-timeout wait-after-sb
}
options {
# cpu-mask on-no-data-accessible
}
disk {
# size max-bio-bvecs on-io-error fencing disk-barrier disk-flushes
# disk-drain md-flushes resync-rate resync-after al-extents
# c-plan-ahead c-delay-target c-fill-target c-max-rate
# c-min-rate disk-timeout
fencing resource-and-stonith;
}
net {
# protocol timeout max-epoch-size max-buffers unplug-watermark
# connect-int ping-int sndbuf-size rcvbuf-size ko-count
# allow-two-primaries cram-hmac-alg shared-secret after-sb-0pri
# after-sb-1pri after-sb-2pri always-asbp rr-conflict
# ping-timeout data-integrity-alg tcp-cork on-congestion
# congestion-fill congestion-extents csums-alg verify-alg
# use-rle
# Protocol "C" tells DRBD not to tell the operating system that
# the write is complete until the data has reach persistent
# storage on both nodes. This is the slowest option, but it is
# also the only one that guarantees consistency between the
# nodes. It is also required for dual-primary, which we will
# be using.
protocol C;
# Tell DRBD to allow dual-primary. This is needed to enable
# live-migration of our servers.
allow-two-primaries yes;
# This tells DRBD what to do in the case of a split-brain when
# neither node was primary, when one node was primary and when
# both nodes are primary. In our case, we'll be running
# dual-primary, so we can not safely recover automatically. The
# only safe option is for the nodes to disconnect from one
# another and let a human decide which node to invalidate. Of
after-sb-0pri discard-zero-changes;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
}
}
====
cat /etc/drbd.d/r0.res
====
# This is the first DRBD resource. If will store the shared file systems and
# the servers designed to run on node 01.
resource r0 {
# These options here are common to both nodes. If for some reason you
# need to set unique values per node, you can move these to the
# 'on <name> { ... }' section.
# This sets the device name of this DRBD resouce.
device /dev/drbd0;
# This tells DRBD what the backing device is for this resource.
disk /dev/sda5;
# This controls the location of the metadata. When "internal" is used,
# as we use here, a little space at the end of the backing devices is
# set aside (roughly 32 MB per 1 TB of raw storage). External metadata
# can be used to put the metadata on another partition when converting
# existing file systems to be DRBD backed, when there is no extra space
# available for the metadata.
meta-disk internal;
# NOTE: this is not required or even recommended with pacemaker. remove
# this options as soon as pacemaker is setup.
startup {
# This tells DRBD to promote both nodes to 'primary' when this
# resource starts. However, we will let pacemaker control this
# so we comment it out, which tells DRBD to leave both nodes
# as secondary when drbd starts.
#become-primary-on both;
}
# NOTE: Later, make it an option in the dashboard to trigger a manual
# verify and/or schedule periodic automatic runs
net {
# TODO: Test performance differences between sha1 and md5
# This tells DRBD how to do a block-by-block verification of
# the data stored on the backing devices. Any verification
# failures will result in the effected block being marked
# out-of-sync.
verify-alg md5;
# TODO: Test the performance hit of this being enabled.
# This tells DRBD to generate a checksum for each transmitted
# packet. If the data received data doesn't generate the same
# sum, a retransmit request is generated. This protects against
# otherwise-undetected errors in transmission, like
# bit-flipping. See:
# http://www.drbd.org/users-guide/s-integrity-check.html
data-integrity-alg md5;
}
# WARNING: Confirm that these are safe when the controller's BBU is
# depleted/failed and the controller enters write-through
# mode.
disk {
# TODO: Test the real-world performance differences gained with
# these options.
# This tells DRBD not to bypass the write-back caching on the
# RAID controller. Normally, DRBD forces the data to be flushed
# to disk, rather than allowing the write-back cachine to
# handle it. Normally this is dangerous, but with BBU-backed
# caching, it is safe. The first option disables disk flushing
# and the second disabled metadata flushes.
disk-flushes no;
md-flushes no;
}
# This sets up the resource on node 01. The name used below must be the
# named returned by "uname -n".
on an-c03n01.alteeve.ca {
# This is the address and port to use for DRBD traffic on this
# node. Multiple resources can use the same IP but the ports
# must differ. By convention, the first resource uses 7788, the
# second uses 7789 and so on, incrementing by one for each
# additional resource.
address 10.10.30.1:7788;
}
on an-c03n02.alteeve.ca {
address 10.10.30.2:7788;
}
}
====
Thanks for the hint, Vladislav!
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?