Loading ...
Sorry, an error occurred while loading the content.

Re: [nslu2-general] Backup thoughts

Expand Messages
  • Marcel Nijenhof
    ... What version of rsync? ... Please verify it with free . A other test would be to run top during the rsync. -- marceln
    Message 1 of 9 , Nov 2, 2007
    • 0 Attachment
      On Wed, 2007-10-31 at 09:35 +0000, Peter Chant wrote:

      > Quite old, V2.3R29-uNSLUng-5.5-beta, I have not been using it for
      > anything but rsync, gallery was too slow (not overclocked this one
      > yet).

      What version of rsync?

      > Swap is listed in fstab, but is not being used (swapon -s) nor can I
      > find anywhere in the init scripts that enables the swap. Hmm need to
      > sort that.

      Please verify it with "free".

      A other test would be to run "top" during the rsync.

      --
      marceln
    • Peter Chant
      ... On the slug: bash-2.05b# rsync --version rsync version 2.6.3 protocol version 28 On slackware 12.0: root@phoenix:~# rsync --version rsync version 2.6.9
      Message 2 of 9 , Nov 6, 2007
      • 0 Attachment
        On Friday 02 November 2007, Marcel Nijenhof wrote:
        > On Wed, 2007-10-31 at 09:35 +0000, Peter Chant wrote:
        > > Quite old, V2.3R29-uNSLUng-5.5-beta, I have not been using it for
        > > anything but rsync, gallery was too slow (not overclocked this one
        > > yet).
        >
        > What version of rsync?

        On the slug:

        bash-2.05b# rsync --version
        rsync version 2.6.3 protocol version 28

        On slackware 12.0:

        root@phoenix:~# rsync --version
        rsync version 2.6.9 protocol version 29

        >
        > > Swap is listed in fstab, but is not being used (swapon -s) nor can I
        > > find anywhere in the init scripts that enables the swap. Hmm need to
        > > sort that.
        >
        > Please verify it with "free".

        BusyBox v0.60.4 (2004.07.01-03:05+0000) Built-in shell (ash)
        Enter 'help' for a list of built-in commands.

        # bash
        bash-2.05b# free
        total used free shared buffers
        Mem: 30520 29796 724 0 11556
        Swap: 56220 20596 35624
        Total: 86740 50392 36348
        bash-2.05b#

        with rsync running, backing up /usr on the big machine - not the particular
        invocation that caused the problem, but I would expect a fairly heavy load.

        Hmm, going down:

        bash-2.05b# free
        total used free shared buffers
        Mem: 30520 30000 520 0 9396
        Swap: 56220 36916 19304
        Total: 86740 66916 19824
        bash-2.05b#

        >
        > A other test would be to run "top" during the rsync.

        On the slug - not particularly high.

        Tasks: 50 total, 2 running, 47 sleeping, 0 stopped, 1 zombie
        Cpu(s): 3.4% user, 2.4% system, 0.0% nice, 94.2% idle
        Mem: 30520k total, 29960k used, 560k free, 13332k buffers
        Swap: 56220k total, 46344k used, 9876k free, 1900k cached

        PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
        4651 root 15 0 26812 1804 192 D 17.0 5.9 0:49.78 rsync
        4676 root 14 0 876 876 688 R 13.6 2.9 0:00.22 top
        10 root 9 0 0 0 0 S 1.7 0.0 10:25.04 usb-storage-1
        1 root 8 0 68 8 8 S 0.0 0.0 0:09.64 init
        2 root 9 0 0 0 0 S 0.0 0.0 0:00.55 keventd
        3 root 19 19 0 0 0 R 0.0 0.0 3:34.82 ksoftirqd_CPU0
        4 root 9 0 0 0 0 S 0.0 0.0 16:46.14 kswapd
        5 root 9 0 0 0 0 S 0.0 0.0 0:05.24 bdflush
        6 root 9 0 0 0 0 S 0.0 0.0 0:01.28 kupdated
        7 root 9 0 0 0 0 S 0.0 0.0 0:00.00 mtdblockd
        8 root 9 0 0 0 0 S 0.0 0.0 0:00.07 khubd
        9 root 14 10 0 0 0 S 0.0 0.0 0:00.00 jffs2_gcd_mtd4
        11 root 9 0 0 0 0 S 0.0 0.0 0:00.00 scsi_eh_1
        29 root 9 0 0 0 0 S 0.0 0.0 0:55.04 kjournald
        53 root 9 0 0 0 0 D 0.0 0.0 0:00.31 ixp425_csr
        54 root 9 0 0 0 0 S 0.0 0.0 0:00.30 ixp425 ixp0
        57 root 9 0 84 0 0 S 0.0 0.0 0:00.02 sh
        bash-2.05b#

        OK, got 30% memory used by rsync a little later on.

        Hmm, seems to get < 30%

        --
        Peter Chant
        http://www.petezilla.co.uk
      • Marcel Nijenhof
        ... The version i used in my test (and the current ipkg version as well): $ rsync --version rsync version 2.6.9 protocol version 29 ... So swap usage
        Message 3 of 9 , Nov 6, 2007
        • 0 Attachment
          On Tue, 2007-11-06 at 20:45 +0000, Peter Chant wrote:

          > On the slug:
          >
          > bash-2.05b# rsync --version
          > rsync version 2.6.3 protocol version 28
          >

          The version i used in my test (and the current ipkg version as well):
          $ rsync --version
          rsync version 2.6.9 protocol version 29


          > # bash
          > bash-2.05b# free
          > total used free shared buffers
          > Mem: 30520 29796 724 0 11556
          > Swap: 56220 20596 35624
          > Total: 86740 50392 36348
          > bash-2.05b#
          >
          > with rsync running, backing up /usr on the big machine - not the
          > particular invocation that caused the problem, but I would expect a
          > fairly heavy load.
          >
          > Hmm, going down:
          >
          > bash-2.05b# free
          > total used free shared buffers
          > Mem: 30520 30000 520 0 9396
          > Swap: 56220 36916 19304
          > Total: 86740 66916 19824

          So swap usage increase 16 mb during this run!
          >
          >
          > >
          > > A other test would be to run "top" during the rsync.
          >
          > On the slug - not particularly high.
          >
          > Tasks: 50 total, 2 running, 47 sleeping, 0 stopped, 1 zombie
          > Cpu(s): 3.4% user, 2.4% system, 0.0% nice, 94.2% idle
          > Mem: 30520k total, 29960k used, 560k free, 13332k buffers
          > Swap: 56220k total, 46344k used, 9876k free, 1900k cached
          >
          > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
          > 4651 root 15 0 26812 1804 192 D 17.0 5.9 0:49.78 rsync
          ^^^^^

          So indeed rsync is using 26 mb of virtual memory.

          This is just the proof we need that rsync is causing the problems!

          Could you try to upgrade to version 2.6.9 which should be installed
          after:
          ipkg update
          ipkg upgrade rsync

          and see if the problem still exists.

          Because i can't reproduce your problem with that version and there is
          a comment in the changelog which says:
          - Reduced the amount of stack memory needed for each level of
          directory recursion by nearly MAXPATHLEN bytes.

          See:
          http://rsync.samba.org/ftp/rsync/old-versions/rsync-2.6.7-NEWS

          --
          marceln
        • Peter Chant
          ... Done. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4743 root 18 0 15548 3680 3580 D 19.5 12.1 0:38.72 rsync OK, while it was
          Message 4 of 9 , Nov 6, 2007
          • 0 Attachment
            On Tuesday 06 November 2007, Marcel Nijenhof wrote:

            > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
            > > 4651 root 15 0 26812 1804 192 D 17.0 5.9 0:49.78 rsync
            >
            > ^^^^^
            >
            > So indeed rsync is using 26 mb of virtual memory.
            >
            > This is just the proof we need that rsync is causing the problems!
            >
            > Could you try to upgrade to version 2.6.9 which should be installed
            > after:

            Done.

            PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
            4743 root 18 0 15548 3680 3580 D 19.5 12.1 0:38.72 rsync

            OK, while it was scanning the slug's hard disk when backing up /usr. 11MB is
            a bit difference on a machine with not much ram. Will report back when the
            backup job completes.

            Pete

            --
            Peter Chant
            http://www.petezilla.co.uk
          • Peter Chant
            ... Looks like it did the trick, just re-running the backup script piping its output to a log file to confirm. Thanks, Pete -- Peter Chant
            Message 5 of 9 , Nov 7, 2007
            • 0 Attachment
              > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
              > 4743 root 18 0 15548 3680 3580 D 19.5 12.1 0:38.72 rsync
              >
              > OK, while it was scanning the slug's hard disk when backing up /usr. 11MB
              > is a bit difference on a machine with not much ram. Will report back when
              > the backup job completes.

              Looks like it did the trick, just re-running the backup script piping its
              output to a log file to confirm.

              Thanks,

              Pete




              --
              Peter Chant
              http://www.petezilla.co.uk
            • Peter Chant
              ... Yep, did the trick. Pete -- Peter Chant http://www.petezilla.co.uk
              Message 6 of 9 , Nov 7, 2007
              • 0 Attachment
                On Wednesday 07 November 2007, Peter Chant wrote:

                > Looks like it did the trick, just re-running the backup script piping its
                > output to a log file to confirm.

                Yep, did the trick.

                Pete


                --
                Peter Chant
                http://www.petezilla.co.uk
              Your message has been successfully submitted and would be delivered to recipients shortly.