[DRBD-user] 0.7.22 does not unload kernel module

Zachár Balázs zachar at direkt-kfki.hu
Wed Jan 10 13:37:44 CET 2007


I have the same problem in that case when the drbd starting to sync...

I found the solution:
In my case it looks like the IBM netcard firmware...

You should turn off the tcp segmentation offload with the ethtool...

You can watch the state with this command:
ethtool -k syncinterface

ethtool -K syncinterface tso off


Regards,
Balázs






Lars Ellenberg írta:
> / 2007-01-10 11:30:23 +0100
> \ Achim Stumpf:
>   
>> Hi list,
>>
>> I have compiled drbd 0.7.22 with fedora kernel 2.6.18-1.2239.fc5 as module.
>> drbd is running fine actually, but i can't unload the module.
>>
>> I have rebooted the server first to have a clean start:
>>
>> [root at clusty6.fra ~]# /etc/init.d/drbd start
>> Starting DRBD resources:    [ d0 s0 n0 ].
>> [root at clusty6.fra ~]# cat /proc/drbd
>> version: 0.7.22 (api:79/proto:74)
>> SVN Revision: 2554 build by root at clusty6.fra, 2007-01-05 12:57:19
>> 0: cs:Connected st:Secondary/Primary ld:Consistent
>>    ns:0 nr:28 dw:28 dr:0 al:0 bm:2 lo:0 pe:0 ua:0 ap:0
>> [root at clusty6.fra ~]# /etc/init.d/drbd stop
>> Stopping all DRBD resourcesERROR: Module drbd is in use
>> .
>> [root at clusty6.fra ~]# cat /proc/drbd
>> version: 0.7.22 (api:79/proto:74)
>> SVN Revision: 2554 build by root at clusty6.fra, 2007-01-05 12:57:19
>> 0: cs:Unconfigured
>> [root at clusty6.fra ~]# rmmod drbd
>> ERROR: Module drbd is in use
>> [root at clusty6.fra ~]# modprobe -r drbd
>> FATAL: Module drbd is in use.
>> [root at clusty6.fra ~]# lsmod
>> Module                  Size  Used by
>> drbd                  141396  1
>> ipv6                  246304  12
>> autofs4                21764  1
>> hidp                   16384  2
>> rfcomm                 38040  0
>> l2cap                  24064  10 hidp,rfcomm
>> bluetooth              50276  5 hidp,rfcomm,l2cap
>> bonding                79348  0
>> dm_mirror              29264  0
>> dm_mod                 57624  1 dm_mirror
>> video                  17412  0
>> sbs                    16576  0
>> i2c_ec                  5760  1 sbs
>> container               5120  0
>> button                  7440  0
>> battery                10884  0
>> asus_acpi              17048  0
>> ac                      5892  0
>> lp                     13256  0
>> sg                     34844  0
>> parport_pc             27684  1
>> serio_raw               7684  0
>> parport                37192  2 lp,parport_pc
>> via686a                17160  0
>> hwmon                   3972  1 via686a
>> i2c_isa                 5888  1 via686a
>> tulip                  50208  0
>> i2c_viapro              9108  0
>> pcspkr                  3712  0
>> i2c_core               21888  4 i2c_ec,via686a,i2c_isa,i2c_viapro
>> ext3                  129800  2
>> jbd                    58408  1 ext3
>> sym53c8xx              72088  3
>> scsi_transport_spi     24960  1 sym53c8xx
>> sd_mod                 20864  20
>> scsi_mod              134056  4 sg,sym53c8xx,scsi_transport_spi,sd_mod
>>
>> The drbd processes are terminated already:
>> [root at clusty6.fra ~]# ps auxf
>> USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
>> root         1  0.1  0.1   2008   668 ?        Ss   11:14   0:00 init [3]
>> root         2  0.0  0.0      0     0 ?        SN   11:14   0:00 [ksoftirqd/0]
>> root         3  0.0  0.0      0     0 ?        S    11:14   0:00 [watchdog/0]
>> root         4  0.0  0.0      0     0 ?        S<   11:14   0:00 [events/0]
>> root         5  0.0  0.0      0     0 ?        S<   11:14   0:00 [khelper]
>> root         6  0.0  0.0      0     0 ?        S<   11:14   0:00 [kthread]
>> root         9  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kblockd/0]
>> root        10  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kacpid]
>> root        72  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [cqueue/0]
>> root        75  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [khubd]
>> root        77  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kseriod]
>> root       136  0.0  0.0      0     0 ?        S    11:14   0:00  \_ [pdflush]
>> root       137  0.0  0.0      0     0 ?        S    11:14   0:00  \_ [pdflush]
>> root       138  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kswapd0]
>> root       139  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [aio/0]
>> root       292  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kpsmoused]
>> root       310  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [scsi_eh_0]
>> root       357  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kjournald]
>> root       398  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kauditd]
>> root       966  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kmirrord]
>> root       987  0.0  0.0      0     0 ?        S<   11:14   0:00  \_ [kjournald]
>> root       423  0.0  0.1   2092   596 ?        S<s  11:14   0:00 /sbin/udevd -d
>> root      1324  0.0  0.1   1752   740 ?        Ss   11:14   0:00 syslogd -m 0
>> root      1327  0.0  0.0   1600   416 ?        Ss   11:14   0:00 klogd -x
>> dbus      1346  0.0  0.2   3092  1104 ?        Ss   11:14   0:00 dbus-daemon --system
>> root      1354  0.0  0.1   2288   900 ?        Ss   11:14   0:00 hcid: processing events
>> root      1357  0.0  0.0   1664   496 ?        Ss   11:14   0:00 sdpd
>> root      1377  0.0  0.0      0     0 ?        S<   11:14   0:00 [krfcommd]
>> root      1408  0.0  0.0   1816   480 ?        Ss   11:14   0:00 /usr/bin/hidd --server
>> root      1490  0.0  0.1   1872   720 ?        Ss   11:14   0:00 /usr/sbin/automount --timeout=60 /net program 
>> /etc/auto.net
>> root      1508  0.0  0.0   1868   344 ?        S    11:14   0:00 /usr/sbin/smartd -q never
>> root      1516  0.0  0.0   1596   456 ?        Ss   11:14   0:00 /usr/sbin/acpid
>> root      1532  0.0  0.2   4988  1108 ?        Ss   11:14   0:00 /usr/sbin/sshd
>> root      1721  0.0  0.4   7832  2476 ?        Ss   11:15   0:00  \_ sshd: root at pts/0
>> root      1723  0.0  0.2   4448  1484 pts/0    Ss   11:15   0:00      \_ -bash
>> root      1856  0.0  0.1   2100   812 pts/0    R+   11:26   0:00          \_ ps auxf
>> root      1563  0.0  0.3   8312  1904 ?        Ss   11:14   0:00 sendmail: accepting connections
>> smmsp     1571  0.0  0.3   7368  1700 ?        Ss   11:14   0:00 sendmail: Queue runner at 01:00:00 for 
>> /var/spool/clientmqueue
>> root      1579  0.0  0.2   3144  1108 ?        Ss   11:14   0:00 crond
>> root      1603  0.0  0.0   2176   464 ?        Ss   11:14   0:00 /usr/sbin/atd
>> 68        1616  0.0  0.6   4844  3168 ?        Ss   11:14   0:00 hald
>> root      1617  0.0  0.1   3136  1016 ?        S    11:14   0:00  \_ hald-runner
>> 68        1623  0.0  0.1   2232   872 ?        S    11:15   0:00      \_ /usr/libexec/hald-addon-acpi
>> root      1649  0.0  0.0   1584   412 tty1     Ss+  11:15   0:00 /sbin/mingetty tty1
>> root      1652  0.0  0.0   1584   412 tty2     Ss+  11:15   0:00 /sbin/mingetty tty2
>> root      1655  0.0  0.0   1588   416 tty3     Ss+  11:15   0:00 /sbin/mingetty tty3
>> root      1658  0.0  0.0   1584   408 tty4     Ss+  11:15   0:00 /sbin/mingetty tty4
>> root      1659  0.0  0.0   1584   448 tty5     Ss+  11:15   0:00 /sbin/mingetty tty5
>> root      1660  0.0  0.0   1584   412 tty6     Ss+  11:15   0:00 /sbin/mingetty tty6
>> root      1661  0.0  0.1   2368  1012 ?        Ss   11:15   0:00 /bin/bash /usr/local/isgbin/syslog_relay
>> root      1716  0.0  0.0   1592   380 ?        S    11:15   0:00  \_ /bin/cat /var/local/syslog_fifo
>> root      1717  0.0  0.0   2368   396 ?        S    11:15   0:00  \_ /bin/bash /usr/local/isgbin/syslog_relay
>>
>> And in the logs:
>> Jan 10 11:16:15 clusty6 kernel: drbd: initialised. Version: 0.7.22 (api:79/proto:74)
>> Jan 10 11:16:15 clusty6 kernel: drbd: SVN Revision: 2554 build by root at clusty6.fra, 2007-01-05 12:57:19
>> Jan 10 11:16:15 clusty6 kernel: drbd: registered as block device major 147
>> Jan 10 11:16:16 clusty6 kernel: drbd0: resync bitmap: bits=1188164 words=37132
>> Jan 10 11:16:16 clusty6 kernel: drbd0: size = 4641 MB (4752656 KB)
>> Jan 10 11:16:16 clusty6 kernel: klogd 1.4.1, ---------- state change ----------
>> Jan 10 11:16:17 clusty6 kernel: drbd0: 0 KB marked out-of-sync by on disk bit-map.
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Found 4 transactions (6 active extents) in activity log.
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbdsetup [1770]: cstate Unconfigured --> StandAlone
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbdsetup [1783]: cstate StandAlone --> Unconnected
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate Unconnected --> WFConnection
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate WFConnection --> WFReportParams
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Handshake successful: DRBD Network Protocol version 74
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Connection established.
>> Jan 10 11:16:17 clusty6 kernel: drbd0: I am(S): 1:00000004:00000001:00000020:00000003:01
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Peer(P): 1:00000004:00000001:00000021:00000003:10
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate WFReportParams --> WFBitMapT
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Secondary/Unknown --> Secondary/Primary
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate WFBitMapT --> SyncTarget
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Resync started as SyncTarget (need to sync 28 KB [7 bits set]).
>> Jan 10 11:16:17 clusty6 kernel: drbd0: Resync done (total 1 sec; paused 0 sec; 28 K/sec)
>> Jan 10 11:16:17 clusty6 kernel: drbd0: drbd0_worker [1771]: cstate SyncTarget --> Connected
>> Jan 10 11:16:42 clusty6 kernel: drbd0: drbdsetup [1800]: cstate Connected --> Unconnected
>> Jan 10 11:16:42 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate Unconnected --> BrokenPipe
>> Jan 10 11:16:42 clusty6 kernel: drbd0: short read expecting header on sock: r=-512
>> Jan 10 11:16:42 clusty6 kernel: drbd0: asender terminated
>> Jan 10 11:16:42 clusty6 kernel: drbd0: worker terminated
>> Jan 10 11:16:42 clusty6 kernel: drbd0: drbd0_receiver [1784]: cstate BrokenPipe --> StandAlone
>> Jan 10 11:16:42 clusty6 kernel: drbd0: Connection lost.
>> Jan 10 11:16:42 clusty6 kernel: drbd0: receiver terminated
>> Jan 10 11:16:42 clusty6 kernel: drbd0: drbdsetup [1800]: cstate StandAlone --> StandAlone
>> Jan 10 11:16:42 clusty6 kernel: drbd0: drbdsetup [1800]: cstate StandAlone --> Unconfigured
>> Jan 10 11:16:42 clusty6 kernel: drbd0: worker terminated
>>     
>
> nothing obvious here.
>
>   
>> Why I can't unload the drbd module.
>>     
>
> "because it still has a reference count != 0"
>  :->
>
>   
>> Do I have to worry about that? What did I miss during compilation?
>>     
>
> I don't know.
> "works for me"... (and many others)
>
> so if you get this to reproduce easily, let me know.
> you may have hit a bug that shows up in your particular setup...
>
>   





More information about the drbd-user mailing list