[DRBD-user] symlink /var/lib/nfs

Alan Robertson alanr at unix.sh
Thu Jun 8 16:17:09 CEST 2006

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.

Todd Denniston wrote:
> Dave Dykstra wrote:
>> On Tue, Jun 06, 2006 at 08:37:07PM -0700, mike wrote:
>>> On 6/6/06, Montse Seisdedos <mseisdedos at mediafusion.es> wrote:
>>>> Hello everybody:
>>>> I'm trying to built a HA nfs server  in a FC4.
>>>> I've been reading about link or not to link /var/lib/nfs to the 
>>>> share fs.
>>>> I did it, but when i try to start nfs i get the following errors
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: can't open /var/lib/nfs/rmtab 
>>>> for
>>>> reading
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: could not open 
>>>> /var/lib/nfs/xtab for
>>>> locking
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: could not open 
>>>> /var/lib/nfs/etab for
>>>> locking
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: can't lock /var/lib/nfs/etab for
>>>> writing
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: could not open 
>>>> /var/lib/nfs/xtab for
>>>> locking
>>>> Jun  6 11:01:17 mfs-01 exportfs[5969]: can't lock /var/lib/nfs/xtab for
>>>> writing
>>>> # ls -l /var/lib/nfs
>>>> lrwxrwxrwx  1 root root 12 jun  6 10:02 /var/lib/nfs -> /u/varlibnfs
>>>> # ls -la /var/lib/nfs/
>>>> total 120
>>>> drwxr-xr-x   4 root    root     4096 jun  6 10:08 .
>>>> drwxr-xr-x  17 root    root    77824 jun  6 10:07 ..
>>>> -rw-r--r--   1 root    root        0 sep 27  2005 etab
>>>> -rw-r--r--   1 root    root        0 sep 27  2005 rmtab
>>>> drwxr-xr-x   2 root    root     4096 sep 27  2005 rpc_pipefs
>>>> drwx------   2 rpcuser rpcuser  4096 sep 27  2005 statd:
>>>> -rw-------   1 root    root        0 s:ep 27  2005 state
>>>> -rw-r--r--   1 root    root        0 sep 27  2005 xtab
>> I think those errors show that you must not be properly stopping and
>> starting all the nfs services.  I'm not very familiar with what the
>> packages are called in FC4, but please post your haresources file and
>> hopefully someone can point out what's wrong with it.  You should only
>> be attempting to access the shared /var/lib/nfs files on the machine
>> that's the active server and so that directory should be available.
>>>> somebody can help me, is really necesary to mv /var/lib/nfs to the 
>>>> share fs?
>>> I was getting weird errors too trying to symlink the shared NFS dir
>>> (I'm using NFS v4 on CentOS 4.3) so DRBD could propogate it.
>>> What I realized (and I could be wrong, but it appears to work) is that
>>> it doesn't *need* to be shared, as long as on failover you restart the
>>> NFS services (portmapper, nfsd, all the RPC crap) - it seems to pick
>>> up where it left off.
>>> The only thing I can think is possibly if it was in the middle of some
>>> process/transaction like writing a large file when it crashed, it
>>> might have been able to persist that, but for my setup, those were
>>> odds I could live with.
>> As far as I have been able to figure out, /var/lib/nfs contains the state
>> of exported filesystems and of locks.  If you're re-exporting everything
>> when you start up the HA service, re-using the exported state from the
>> previous active server doesn't really help much.  In theory you should be
>> able to keep locks across fail-overs, but in practice I've not been able
>> to get that to work.  I was experimenting with that in order to figure 
>> out
>> what exactly should be on the http://linux-ha.org/HaNFS web page but I 
>> got
>> stuck on that problem so I asked for help on the linux-ha mailing list 
>> and
>> stopped working on the web page.  I never did figure out an answer and
>> then I got too busy with other things and never got back to it.  So in
>> practice I don't think it's really necessary to have the /var/lib/nfs
>> symlinks even though that's been the advice for the linux-ha+drbd+NFS
>> configuration for a long time.  I really should clean up that page.
>> - Dave Dykstra
> One reason for keeping /var/lib/nfs as a sym-link, besides locking, is 
> that without it clients who have the export mounted while the fallover 
> happens will start getting stale NFS handle errors when the secondary 
> takes over. The other big reason to do it is that if a client not only 
> has it mounted but is transferring a file while the fallover happens, 
> with the simlink the client just picks up where it stopped getting ACKs 
> back, with out the link the client just hangs/crashes/errors.
> I suggest don't change that recommendation!!!.

Although Dave reports that his locks don't fail over correctly, we had 
no trouble with that when we followed the guidelines now on the web 
site.  So, I second this opinion.

     Alan Robertson <alanr at unix.sh>

"Openness is the foundation and preservative of friendship...  Let me 
claim from you at all times your undisguised opinions." - William 

More information about the drbd-user mailing list