Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.
Hi Brad,
On Sat, Dec 10, 2005 at 07:46:06AM -0500, Brad Barnett wrote:
> On Wed, 7 Dec 2005 10:45:20 +0100
> raoul at sgs.dfn.de (Raoul Borenius) wrote:
>
> > Hi,
> >
> > has anyone succeeded in setting up multiple nfs-exported drbd-volumes
> > without getting the 'Stale NFS file handle'-error during failover?
> >
> > Following http://linux-ha.org/DRBD_2fNFS I can export one drbd-volume
> > with symlinked /var/lib/nfs which works perfectly without error during
> > failover. But what do I need to do if I have two or more drbd-volumes on
> > my nfs-server (which will be exported to different clients)?
>
> I have to ask this, just in case you may be doing something strange. ;)
> If you have a large raid, there is no need to setup a separate partition
> for each NFS export. That is, if you have four NFS exports, they can all
> live in the same directory on your raid/drbd device.. and be nfs exported
> from there.
That's true of course. But we've had some bad experiences with multiple
simultaneous disk-failures so we're trying to do without raid.
> Now, I know you may have four different drives, non raided.. and that you
> may be exporting them each.. but I thought I would mention the above, just
> in case you thought each nfs export needed its own partition.
>
> >
> > /var/lib/nfs can only be symlinked to one of the volumes which
> > means that I would get a 'Stale NFS file handle'-error during failover
> > on all the other volumes.
> >
> > I've tried to set up /var/lib/nfs as drbd0 and then my data-volumes as
> > drbd1, drbd2 etc. but that did not help.
> >
>
> It should help. Is it that the drbd0 partition is not ready soon enough?
> I would think that heartbeat failover would mean that all drbd partitions
> are ready, before NFS is started..
>
> Do you know why this failed? It seems like a logical idea...
Here ist my haresources with only one data-volume:
b1 drbddisk::varlibnfs \
drbddisk::mail \
Filesystem::/dev/drbd0::/var/lib/nfs::ext3 \
Filesystem::/dev/drbd1::/srv/nfs/mail::ext3 \
killnfsd \
nfs-common \
nfs-kernel-server \
sleep::3 \
IPaddr::194.95.249.20/24/eth0
> If it looks like the drbd failover worked, that is, the /var/lib/nfs
> symlink was pointed to drbd0, and drbd0 was there before nfs started.. it
> may be something else that is causing stales during failover.
Maybe it's just that there needs to be some sort of sync between
/var/lib/nfs and the actual exportet files? And if that is put on
different drbds there might always be some kind of inconsistency between
the two...?
> Personally, I moved from a live, non-drbd setup to a drbd setup, so I had
> several non-friendly mount options on my nfs clients that caused the
> failover to fail in such a way.
>
> What are your nfs export and nfs client options, for mounting? Double
> check these, for sure...
I'm using nfs-defaults:
/etc/exports:
/srv/nfs/mail 194.95.249.0/24(rw,sync)
On client:
raoul at b5:~$ mount
/dev/sda1 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
tmpfs on /tmp type tmpfs (rw)
tmpfs on /var/tmp type tmpfs (rw)
tmpfs on /dev type tmpfs (rw,size=10M,mode=0755)
nfs:/srv/nfs/mail on /mnt type nfs (rw,addr=194.95.249.20)
raoul at b5:~$
> > I've been reading the list-archives and googling around but haven't
> > found anything that pointed me to where the problem lies.
> >
> > Is it not possible to use more than one volume for nfs-export?
Would be nice to hear if anyone has succeeded in using more than one
drbd and nfs.
At the moment I'm about to give up and set up software-raid and one
drbd on top of that.
Thanks for your help anyway!
Regards
Raoul