Loading ...
Sorry, an error occurred while loading the content.
 

NFSD: unable to find recovery directory /var/lib/nfs/v4recovery

Expand Messages
  • slackline
    Hi, Unfortunately my slug got knocked off the shelf when my (ill) cat failed to jump onto the shelf. As a result it had a power-cycle without correct shutdown
    Message 1 of 5 , Jan 8, 2010
      Hi,

      Unfortunately my slug got knocked off the shelf when my (ill) cat failed to jump onto the shelf. As a result it had a power-cycle without correct shutdown and I now find I'm having trouble exporting (and mounting from a client) my NFS directories.

      I see in dmesg....

      eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
      RPC: Registered udp transport module.
      RPC: Registered tcp transport module.
      Installing knfsd (copyright (C) 1996 okir@...).
      NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
      NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
      NFSD: starting 90-second grace period


      and from the client I get permission denied on all directories...

      # /etc/init.d/netmount restart
      * Caching service dependencies ... [ ok ]
      * Unmounting network filesystems ... [ ok ]
      * Mounting network filesystems ...
      mount.nfs: access denied by server while mounting slug:/mnt/torrents
      mount.nfs: access denied by server while mounting slug:/mnt/albums
      mount.nfs: access denied by server while mounting slug:/mnt/music
      mount.nfs: access denied by server while mounting slug:/mnt/video
      mount.nfs: access denied by server while mounting slug:/mnt/pics
      mount.nfs: access denied by server while mounting slug:/mnt/doc
      mount.nfs: access denied by server while mounting slug:/mnt/work1
      mount.nfs: access denied by server while mounting slug:/mnt/work2
      mount.nfs: access denied by server while mounting slug:/mnt/ref
      * Could not mount all network filesystems [ !! ]


      I found a suggestion of adding the 'nolock' option to /etc/exports but I'm told this isn't recognised (and its not listed in the man page for /etc/exports either).

      Currently my /etc/exports looks like...

      # /etc/exports: NFS file systems being exported. See exports(5).
      ## /usr/portage
      /mnt/portage 192.168.1.0/255.255.255.0(rw,async,subtree_check,no_root_squash)
      ## 250Gb LaCie
      ## pics1
      /mnt/torrents 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## music1
      /mnt/albums 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## 500Gb LaCie
      ## music2
      /mnt/music 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## video
      /mnt/video 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## pics2
      /mnt/pics 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## doc
      /mnt/doc 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## misc
      /mnt/work1 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## work
      /mnt/work2 192.168.1.0/255.255.255.0(rw,async,subtree_check)
      ## ref
      /mnt/ref 192.168.1.0/255.255.255.0(rw,async,subtree_check)


      The versions of NFS (and modules) I have installed are...

      # opkg list_installed | grep nfs
      kernel-module-nfs - 2.6.27.8+svnr1085-r3 -
      kernel-module-nfsd - 2.6.27.8+svnr1085-r3 -
      nfs-utils - 1.1.2-3 -


      I get the impression that there is a lock on NFS exports due to the unclean power-cycle, but have no idea how to get round this.

      Advice appreciated,

      Cheers

      Neil
    • Mike Westerhof (mwester)
      ... Red herring. SlugOS 5.3 doesn t do NFS v4 (it supports V3 and V2), so this message is just noise (we probably should have disabled V4 support altogether,
      Message 2 of 5 , Jan 10, 2010
        slackline wrote:
        > Hi,
        >
        > Unfortunately my slug got knocked off the shelf when my (ill) cat failed to jump onto the shelf. As a result it had a power-cycle without correct shutdown and I now find I'm having trouble exporting (and mounting from a client) my NFS directories.
        >
        > I see in dmesg....
        >
        > eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
        > RPC: Registered udp transport module.
        > RPC: Registered tcp transport module.
        > Installing knfsd (copyright (C) 1996 okir@...).
        > NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
        > NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
        > NFSD: starting 90-second grace period
        >
        Red herring. SlugOS 5.3 doesn't do NFS v4 (it supports V3 and V2), so
        this message is just noise (we probably should have disabled V4 support
        altogether, but the effort to do so didn't seem worth the time, since
        it's just a cosmetic message).

        > and from the client I get permission denied on all directories...
        >
        > # /etc/init.d/netmount restart
        > * Caching service dependencies ... [ ok ]
        > * Unmounting network filesystems ... [ ok ]
        > * Mounting network filesystems ...
        > mount.nfs: access denied by server while mounting slug:/mnt/torrents
        > mount.nfs: access denied by server while mounting slug:/mnt/albums
        > mount.nfs: access denied by server while mounting slug:/mnt/music
        > mount.nfs: access denied by server while mounting slug:/mnt/video
        > mount.nfs: access denied by server while mounting slug:/mnt/pics
        > mount.nfs: access denied by server while mounting slug:/mnt/doc
        > mount.nfs: access denied by server while mounting slug:/mnt/work1
        > mount.nfs: access denied by server while mounting slug:/mnt/work2
        > mount.nfs: access denied by server while mounting slug:/mnt/ref
        > * Could not mount all network filesystems [ !! ]
        >
        >
        > I found a suggestion of adding the 'nolock' option to /etc/exports but I'm told this isn't recognised (and its not listed in the man page for /etc/exports either).
        >
        > Currently my /etc/exports looks like...
        >
        > # /etc/exports: NFS file systems being exported. See exports(5).
        > ## /usr/portage
        > /mnt/portage 192.168.1.0/255.255.255.0(rw,async,subtree_check,no_root_squash)
        > ## 250Gb LaCie
        > ## pics1
        > /mnt/torrents 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## music1
        > /mnt/albums 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## 500Gb LaCie
        > ## music2
        > /mnt/music 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## video
        > /mnt/video 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## pics2
        > /mnt/pics 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## doc
        > /mnt/doc 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## misc
        > /mnt/work1 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## work
        > /mnt/work2 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        > ## ref
        > /mnt/ref 192.168.1.0/255.255.255.0(rw,async,subtree_check)
        >
        >
        > The versions of NFS (and modules) I have installed are...
        >
        > # opkg list_installed | grep nfs
        > kernel-module-nfs - 2.6.27.8+svnr1085-r3 -
        > kernel-module-nfsd - 2.6.27.8+svnr1085-r3 -
        > nfs-utils - 1.1.2-3 -
        >
        >
        > I get the impression that there is a lock on NFS exports due to the unclean power-cycle, but have no idea how to get round this.
        >
        There's no lock involved; there is no state maintained for NFS mounts
        themselves (and there's actually no locking maintained unless you are
        running the lockd, but even so it won't prevent a mount from happening).

        Look for a permissions issue. I would wonder if, at the time your
        exports are being processed, those mount points in /mnt are actually
        mounted yet?

        -Mike (mwester)
      • slackline
        Hi Mike, Thanks for the reply. ... Ok, duly ignored. ... I ve tried restarting the nfsserver on the slug and still get the same error, and the file systems are
        Message 3 of 5 , Jan 10, 2010
          Hi Mike,

          Thanks for the reply.

          --- In nslu2-linux@yahoogroups.com, "Mike Westerhof (mwester)" <mwester@...> wrote:
          >
          > >
          > > I see in dmesg....
          > >
          > > eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
          > > RPC: Registered udp transport module.
          > > RPC: Registered tcp transport module.
          > > Installing knfsd (copyright (C) 1996 okir@...).
          > > NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
          > > NFSD: unable to find recovery directory /var/lib/nfs/v4recovery
          > > NFSD: starting 90-second grace period
          > >
          > Red herring. SlugOS 5.3 doesn't do NFS v4 (it supports V3 and V2), so
          > this message is just noise (we probably should have disabled V4 support
          > altogether, but the effort to do so didn't seem worth the time, since
          > it's just a cosmetic message).

          Ok, duly ignored.

          > There's no lock involved; there is no state maintained for NFS mounts
          > themselves (and there's actually no locking maintained unless you are
          > running the lockd, but even so it won't prevent a mount from happening).
          >
          > Look for a permissions issue. I would wonder if, at the time your
          > exports are being processed, those mount points in /mnt are actually
          > mounted yet?


          I've tried restarting the nfsserver on the slug and still get the same error, and the file systems are mounted...


          root@slug:/var/log# df -h
          Filesystem 1K-blocks Used Available Use% Mounted on
          /dev/root 6400 5472 928 86% /initrd
          /dev/sdb3 9843308 2935996 6407292 31% /
          /dev/sdb3 9843308 2935996 6407292 31% /dev/.static/dev
          udev 2048 60 1988 3% /dev
          /dev/sdb1 241116 173813 54855 76% /mnt/backup
          /dev/sdb5 9841097 6499703 2841373 70% /mnt/portage
          /dev/sdb6 109969760 81428728 22954832 78% /mnt/torrents
          /dev/sdb7 109969760 30633912 73749648 29% /mnt/albums
          /dev/sda1 196870980 155003496 31867024 83% /mnt/music
          /dev/sda2 98435476 87451408 5983840 94% /mnt/video
          /dev/sda3 98435476 93433356 1892 100% /mnt/pics
          /dev/sda5 19694836 14750992 3943400 79% /mnt/doc
          /dev/sda6 19694836 3840028 14854364 21% /mnt/work1
          /dev/sda7 29538432 6120760 21917204 22% /mnt/work2
          /dev/sda8 18050192 3529108 13604176 21% /mnt/ref
          tmpfs 15120 0 15120 0% /var/volatile
          tmpfs 15120 0 15120 0% /dev/shm


          I thought I'd check from my laptop and I get...

          mount.nfs: mount to NFS server 'slug:/mnt/ref' failed: RPC Error: Program not registered

          So I set about checking that RPC was running properly on the slug and could be seen from the client(s).

          On the slug I get...

          root@slug:/var/log# rpcinfo -p
          program vers proto port
          100000 2 tcp 111 portmapper
          100021 1 udp 47279 nlockmgr
          100021 3 udp 47279 nlockmgr
          100021 4 udp 47279 nlockmgr
          100003 2 udp 2049 nfs
          100003 3 udp 2049 nfs
          100003 4 udp 2049 nfs
          100021 1 tcp 48775 nlockmgr
          100021 3 tcp 48775 nlockmgr
          100021 4 tcp 48775 nlockmgr
          100003 2 tcp 2049 nfs
          100003 3 tcp 2049 nfs
          100003 4 tcp 2049 nfs
          100000 2 udp 111 portmapper

          On the clients I get the same information...

          # rpcinfo -p slug
          program vers proto port
          100000 2 tcp 111 portmapper
          100021 1 udp 47279 nlockmgr
          100021 3 udp 47279 nlockmgr
          100021 4 udp 47279 nlockmgr
          100003 2 udp 2049 nfs
          100003 3 udp 2049 nfs
          100003 4 udp 2049 nfs
          100021 1 tcp 48775 nlockmgr
          100021 3 tcp 48775 nlockmgr
          100021 4 tcp 48775 nlockmgr
          100003 2 tcp 2049 nfs
          100003 3 tcp 2049 nfs
          100003 4 tcp 2049 nfs
          100000 2 udp 111 portmapper

          Which suggests to me that things can be seen on the slug server fine. I've tried restarting the nfs mounts on the clients and server in turn, but no joy at all.

          So I'm still stumped by this unfortunately.

          Cheers

          Neil
        • Mike Westerhof (mwester)
          ... I don t think they can be seen fine -- where s your mountd? The nfsserver startup script (in /etc/init.d) starts the /usr/sbin/mountd daemon -- your s
          Message 4 of 5 , Jan 10, 2010
            slackline wrote:
            > # rpcinfo -p slug
            > program vers proto port
            > 100000 2 tcp 111 portmapper
            > 100021 1 udp 47279 nlockmgr
            > 100021 3 udp 47279 nlockmgr
            > 100021 4 udp 47279 nlockmgr
            > 100003 2 udp 2049 nfs
            > 100003 3 udp 2049 nfs
            > 100003 4 udp 2049 nfs
            > 100021 1 tcp 48775 nlockmgr
            > 100021 3 tcp 48775 nlockmgr
            > 100021 4 tcp 48775 nlockmgr
            > 100003 2 tcp 2049 nfs
            > 100003 3 tcp 2049 nfs
            > 100003 4 tcp 2049 nfs
            > 100000 2 udp 111 portmapper
            >
            > Which suggests to me that things can be seen on the slug server fine. I've tried restarting the nfs mounts on the clients and server in turn, but no joy at all.
            I don't think they can be seen fine -- where's your mountd? The
            nfsserver startup script (in /etc/init.d) starts the /usr/sbin/mountd
            daemon -- your's isn't there in the rpc listing. If you can figure out
            where it went, or why it isn't starting, you'll have a good idea what's
            going on.

            -Mike (mwester)
          • slackline
            ... Nail hit on the head! I ve no idea why, as I d restarted it several times, but this time I got... # /etc/init.d/nfsserver restart stopping mountd: done
            Message 5 of 5 , Jan 10, 2010
              --- In nslu2-linux@yahoogroups.com, "Mike Westerhof (mwester)" <mwester@...> wrote:
              >
              > slackline wrote:
              > > # rpcinfo -p slug
              > > program vers proto port
              > > 100000 2 tcp 111 portmapper
              > > 100021 1 udp 47279 nlockmgr
              > > 100021 3 udp 47279 nlockmgr
              > > 100021 4 udp 47279 nlockmgr
              > > 100003 2 udp 2049 nfs
              > > 100003 3 udp 2049 nfs
              > > 100003 4 udp 2049 nfs
              > > 100021 1 tcp 48775 nlockmgr
              > > 100021 3 tcp 48775 nlockmgr
              > > 100021 4 tcp 48775 nlockmgr
              > > 100003 2 tcp 2049 nfs
              > > 100003 3 tcp 2049 nfs
              > > 100003 4 tcp 2049 nfs
              > > 100000 2 udp 111 portmapper
              > >
              > > Which suggests to me that things can be seen on the slug server fine. I've tried restarting the nfs mounts on the clients and server in turn, but no joy at all.
              > I don't think they can be seen fine -- where's your mountd? The
              > nfsserver startup script (in /etc/init.d) starts the /usr/sbin/mountd
              > daemon -- your's isn't there in the rpc listing. If you can figure out
              > where it went, or why it isn't starting, you'll have a good idea what's
              > going on.
              >

              Nail hit on the head!

              I've no idea why, as I'd restarted it several times, but this time I got...

              # /etc/init.d/nfsserver restart
              stopping mountd: done
              stopping statd: done
              starting mountd: done
              starting statd: done
              # rpcinfo -p
              program vers proto port
              100000 2 tcp 111 portmapper
              100021 1 udp 47279 nlockmgr
              100021 3 udp 47279 nlockmgr
              100021 4 udp 47279 nlockmgr
              100003 2 udp 2049 nfs
              100003 3 udp 2049 nfs
              100003 4 udp 2049 nfs
              100021 1 tcp 48775 nlockmgr
              100021 3 tcp 48775 nlockmgr
              100021 4 tcp 48775 nlockmgr
              100003 2 tcp 2049 nfs
              100003 3 tcp 2049 nfs
              100003 4 tcp 2049 nfs
              100000 2 udp 111 portmapper
              100005 1 udp 44299 mountd
              100005 1 tcp 52699 mountd
              100005 2 udp 44299 mountd
              100005 2 tcp 52699 mountd
              100005 3 udp 44299 mountd
              100005 3 tcp 52699 mountd
              100024 1 udp 36316 status
              100024 1 tcp 48092 status

              I can now mount from the laptop where I was getting "RPC Error: Program not registered", but strangely still get the permission denied error from my desktop despite being able to see the slug running with mountd...

              # /etc/init.d/netmount restart
              * Caching service dependencies ... [ ok ]
              * Unmounting network filesystems ... [ ok ]
              * Mounting network filesystems ...
              mount.nfs: access denied by server while mounting slug:/mnt/torrents
              mount.nfs: access denied by server while mounting slug:/mnt/albums
              mount.nfs: access denied by server while mounting slug:/mnt/music
              mount.nfs: access denied by server while mounting slug:/mnt/video
              mount.nfs: access denied by server while mounting slug:/mnt/pics
              mount.nfs: access denied by server while mounting slug:/mnt/doc
              mount.nfs: access denied by server while mounting slug:/mnt/work1
              mount.nfs: access denied by server while mounting slug:/mnt/work2
              mount.nfs: access denied by server while mounting slug:/mnt/ref
              * Could not mount all network filesystems [ !! ]
              darwin pics # rpcinfo -p slug
              program vers proto port service
              100000 2 tcp 111 portmapper
              100021 1 udp 47279 nlockmgr
              100021 3 udp 47279 nlockmgr
              100021 4 udp 47279 nlockmgr
              100003 2 udp 2049 nfs
              100003 3 udp 2049 nfs
              100003 4 udp 2049 nfs
              100021 1 tcp 48775 nlockmgr
              100021 3 tcp 48775 nlockmgr
              100021 4 tcp 48775 nlockmgr
              100003 2 tcp 2049 nfs
              100003 3 tcp 2049 nfs
              100003 4 tcp 2049 nfs
              100000 2 udp 111 portmapper
              100005 1 udp 44299 mountd
              100005 1 tcp 52699 mountd
              100005 2 udp 44299 mountd
              100005 2 tcp 52699 mountd
              100005 3 udp 44299 mountd
              100005 3 tcp 52699 mountd
              100024 1 udp 36316 status
              100024 1 tcp 48092 status

              The /etc/fstab entries are identical between the two....

              ### NSLU2 under SlugOSBE
              ## /media/sda (250Gb)
              #slug:/mnt/portage /usr/portage nfs auto,rw,users 0 0
              slug:/mnt/torrents /mnt/slug/torrents nfs auto,rw,users 0 0
              slug:/mnt/albums /mnt/slug/albums nfs auto,rw,users 0 0
              ## /mnt/sdb (500Gb)
              slug:/mnt/music /mnt/slug/music nfs auto,rw,users 0 0
              slug:/mnt/video /mnt/slug/video nfs auto,rw,users 0 0
              slug:/mnt/pics /mnt/slug/pics2 nfs auto,rw,users 0 0
              slug:/mnt/doc /mnt/slug/doc nfs auto,rw,users 0 0
              slug:/mnt/work1 /mnt/slug/misc nfs auto,rw,users 0 0
              slug:/mnt/work2 /mnt/slug/work nfs auto,rw,users 0 0
              slug:/mnt/ref /mnt/slug/ref nfs auto,rw,users 0 0

              ...whilst the /etc/exports on the server obviously doesn't vary...

              # more /etc/exports
              # /etc/exports: NFS file systems being exported. See exports(5).
              ## 250Gb LaCie
              ## pics1
              /mnt/torrents 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## music1
              /mnt/albums 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## 500Gb LaCie
              ## music2
              /mnt/music 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## video
              /mnt/video 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## pics2
              /mnt/pics 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## doc
              /mnt/doc 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## misc
              /mnt/work1 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## work
              /mnt/work2 192.168.1.0/255.255.255.0(rw,async,subtree_check)
              ## ref
              /mnt/ref 192.168.1.0/255.255.255.0(rw,async,subtree_check)

              Its strange that an improper power-cycle has wrecked such havoc!

              Thank you very much for your suggestions so far, feels very close now.

              Neil
            Your message has been successfully submitted and would be delivered to recipients shortly.