[DRBD-user] Why need both semaphore and spinlock to protect work queue

[wang xuchen]

Hi All,

I appreciate if somebody can tell me why spinlock and semaphore are utilized
to protect work queue in drbd worker thread. I have some performance issue
with regard to the use of this semaphore. Specifically, down_interruptible
takes too long to be waked up. Here is the detail:

In drbd_worker.c, function int drbd_worker(struct drbd_thread *thi).

int drbd_worker(struct drbd_thread *thi)
{
....
                if (down_trylock(&mdev->data.work.s)) {
                        mutex_lock(&mdev->data.mutex);
                        if (mdev->data.socket && !mdev->net_conf->no_cork)
                                drbd_tcp_uncork(mdev->data.socket);
                        mutex_unlock(&mdev->data.mutex);

                        intr = down_interruptible(&mdev->data.work.s);

                        mutex_lock(&mdev->data.mutex);
                        if (mdev->data.socket  && !mdev->net_conf->no_cork)
                                drbd_tcp_cork(mdev->data.socket);
                        mutex_unlock(&mdev->data.mutex);
                }

....
               w = NULL;
                spin_lock_irq(&mdev->data.work.q_lock);
                ERR_IF(list_empty(&mdev->data.work.q)) {
                        /* something terribly wrong in our logic.
                         * we were able to down() the semaphore,
                         * but the list is empty... doh.
                         *
                         * what is the best thing to do now?
                         * try again from scratch, restarting the receiver,
                         * asender, whatnot? could break even more ugly,
                         * e.g. when we are primary, but no good local data.
                         *
                         * I'll try to get away just starting over this
loop.
                         */
                        spin_unlock_irq(&mdev->data.work.q_lock);
                        continue;
                }
...
}

I tried to get rid of semaphore in above as well as in drbd_queue_work in
drbd_ini.h. Somehow, the code breaks ugly. I guess I still don`t get the
design rational here. Any help is appreciated.

Thanks
Ben
Commit yourself to constant self-improvement
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linbit.com/pipermail/drbd-user/attachments/20100929/44872365/attachment.htm>