Loading ...
Sorry, an error occurred while loading the content.
 

Slow rsync speed between usb drives

Expand Messages
  • docbillnet
    Hello, I just purchased a couple of NSLU2 from tigerdirect.ca, and I have followed the instructions for installing Linux on one. Overall, it seems to be
    Message 1 of 18 , Mar 26, 2008
      Hello,

      I just purchased a couple of NSLU2 from tigerdirect.ca, and I have
      followed the instructions for installing Linux on one. Overall, it
      seems to be working well, especially since I cut the resistor.
      However, I wanted to backup data from three 500 GB drives to a 1.5 TB
      drive. However, when I used the --progress option with rsync, it
      reported the transfer speed is 2.0 MB/s. After a quick back of the
      envelope calculation, I concluded it will take 8 days to copy my data
      from the three drives.

      So, I shutdown the NSLU2, and plugged the same drives and USB hub into
      my laptop. I issued the same rsync command, and my transfer rate is
      20 MB/s.

      Is this a typical transfer speed with an NSLU2? If not, how do I go
      about determining the problem and speeding up the transfer? From what
      I've seen in this forum, I would expect a transfer speed between
      drives of about 6 MB/s, but I can't be certain since the messages were
      all related to how long it took to transfer files between a usb drive
      and the network. Perhaps, the USB bus is much slower when it is both
      reading and writing at the same time. Or perhaps rsync has too much
      overhead, and I need to use a different program to copy data.

      Bill
    • Carl Lowenstein
      ... Exactly what command did you give rsync? Did you possibly enable either encryption or compression? Neither of these are necessary for a local transfer,
      Message 2 of 18 , Mar 26, 2008
        On 3/26/08, docbillnet <yahoo@...> wrote:
        >
        > Hello,
        >
        > I just purchased a couple of NSLU2 from tigerdirect.ca, and I have
        > followed the instructions for installing Linux on one. Overall, it
        > seems to be working well, especially since I cut the resistor.
        > However, I wanted to backup data from three 500 GB drives to a 1.5 TB
        > drive. However, when I used the --progress option with rsync, it
        > reported the transfer speed is 2.0 MB/s. After a quick back of the
        > envelope calculation, I concluded it will take 8 days to copy my data
        > from the three drives.

        Exactly what command did you give rsync?

        Did you possibly enable either encryption or compression? Neither of
        these are necessary for a local transfer, and both of them are
        computationally intensive. The NSLU2 has a CPU of very little brain.


        > So, I shutdown the NSLU2, and plugged the same drives and USB hub into
        > my laptop. I issued the same rsync command, and my transfer rate is
        > 20 MB/s.

        You couild do the initial backup with the laptop, taking one day
        according to your calculations. Updating the backup with the NSLU2
        and rsync will take much less time, because only the changes are
        transferred.

        carl
        --
        carl lowenstein
        marine physical lab, u.c. san diego
        clowenstein@...
      • docbillnet
        It looks like in general write speeds on the NSLU2 are being slow: docbill@LKG6BE74A:~$ dd if=/dev/zero of=/tmp/x bs=4096 count=100000 100000+0 records in
        Message 3 of 18 , Mar 26, 2008
          It looks like in general write speeds on the NSLU2 are being slow:

          docbill@LKG6BE74A:~$ dd if=/dev/zero of=/tmp/x bs=4096 count=100000
          100000+0 records in
          100000+0 records out
          409600000 bytes (410 MB) copied, 197.421 seconds, 2.1 MB/s

          According to
          http://www.nslu2-linux.org/wiki/Unslung/SolvePerformanceProblems

          I should be seeing about 10 MB/s. Although, I notice the dd on that
          page only copies 10M, which is not enough for a proper though put
          test. Particularly if none of the data synced to disk during the write.

          Reading a disk is not much faster:

          docbill@LKG6BE74A:/proc/sys$ dd if=/tmp/x of=/dev/null bs=4096
          100000+0 records in
          100000+0 records out
          409600000 bytes (410 MB) copied, 115.896 seconds, 3.5 MB/s


          I don't see any useful log entries with dmesg.


          --- In nslu2-general@yahoogroups.com, "Carl Lowenstein"
          <clowenstein@...> wrote:
          >
          > On 3/26/08, docbillnet <yahoo@...> wrote:
          > >
          > > Hello,
          > >
          > > I just purchased a couple of NSLU2 from tigerdirect.ca, and I have
          > > followed the instructions for installing Linux on one. Overall, it
          > > seems to be working well, especially since I cut the resistor.
          > > However, I wanted to backup data from three 500 GB drives to a 1.5 TB
          > > drive. However, when I used the --progress option with rsync, it
          > > reported the transfer speed is 2.0 MB/s. After a quick back of the
          > > envelope calculation, I concluded it will take 8 days to copy my data
          > > from the three drives.
          >
          > Exactly what command did you give rsync?
          >
          > Did you possibly enable either encryption or compression? Neither of
          > these are necessary for a local transfer, and both of them are
          > computationally intensive. The NSLU2 has a CPU of very little brain.
          >
          >
          > > So, I shutdown the NSLU2, and plugged the same drives and USB hub
          into
          > > my laptop. I issued the same rsync command, and my transfer rate is
          > > 20 MB/s.
          >
          > You couild do the initial backup with the laptop, taking one day
          > according to your calculations. Updating the backup with the NSLU2
          > and rsync will take much less time, because only the changes are
          > transferred.
          >
          > carl
          > --
          > carl lowenstein
          > marine physical lab, u.c. san diego
          > clowenstein@...
          >
        • Mike (mwester)
          ... What firmware did you install? How are the drives mounted (can you provide the output from the mount command)? What exactly is the rsync command you
          Message 4 of 18 , Mar 26, 2008
            docbillnet wrote:
            > Hello,
            >
            > I just purchased a couple of NSLU2 from tigerdirect.ca, and I have
            > followed the instructions for installing Linux on one. Overall, it
            > seems to be working well, especially since I cut the resistor.
            > However, I wanted to backup data from three 500 GB drives to a 1.5 TB
            > drive. However, when I used the --progress option with rsync, it
            > reported the transfer speed is 2.0 MB/s. After a quick back of the
            > envelope calculation, I concluded it will take 8 days to copy my data
            > from the three drives.
            >
            > So, I shutdown the NSLU2, and plugged the same drives and USB hub into
            > my laptop. I issued the same rsync command, and my transfer rate is
            > 20 MB/s.
            >
            > Is this a typical transfer speed with an NSLU2? If not, how do I go
            > about determining the problem and speeding up the transfer? From what
            > I've seen in this forum, I would expect a transfer speed between
            > drives of about 6 MB/s, but I can't be certain since the messages were
            > all related to how long it took to transfer files between a usb drive
            > and the network. Perhaps, the USB bus is much slower when it is both
            > reading and writing at the same time. Or perhaps rsync has too much
            > overhead, and I need to use a different program to copy data.
            >
            > Bill

            What firmware did you install?
            How are the drives mounted (can you provide the output from the mount
            command)?
            What exactly is the rsync command you issued?

            Mike (mwester)
          • docbillnet
            ... debian-4.0r3: di-nslu2.bin ... At the time I had five drives mounted. However, since I know have those drives plugged into my laptop, I am currently
            Message 5 of 18 , Mar 26, 2008
              --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@...>
              wrote:
              > What firmware did you install?

              debian-4.0r3: di-nslu2.bin

              > How are the drives mounted (can you provide the output from the mount
              > command)?

              At the time I had five drives mounted. However, since I know have
              those drives plugged into my laptop, I am currently testing and
              producing the same problem with a single flash drive.

              LKG6BE74A:~# mount
              /dev/sda1 on / type ext3 (rw,errors=remount-ro)
              tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
              proc on /proc type proc (rw,noexec,nosuid,nodev)
              sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
              procbususb on /proc/bus/usb type usbfs (rw)
              udev on /dev type tmpfs (rw,mode=0755)
              tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
              devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
              nfsd on /proc/fs/nfsd type nfsd (rw)
              rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)

              > What exactly is the rsync command you issued?

              rsync --progress -aHSA . /mnt/share/Series/.

              Although, I produce the same slow speeds with dd and hdparm:

              LKG6BE74A:~# hdparm -t -T /dev/sda

              /dev/sda:
              Timing cached reads: 96 MB in 2.02 seconds = 47.46 MB/sec
              Timing buffered disk reads: 10 MB in 3.12 seconds = 3.21 MB/sec

              I gave the sample dd commands in a separate message.

              Basically, I can read data at about 3 MB/s and I can write data at 2
              MB/s. Regardless of which disks I use.

              Bill
            • docbillnet
              I found the cause of the problem, and the solution. When you clip the resistor to speed up the CPU, the voltage to the USB ports drops below specification.
              Message 6 of 18 , Mar 27, 2008
                I found the cause of the problem, and the solution.

                When you clip the resistor to speed up the CPU, the voltage to the USB
                ports drops below specification. This causes devices to run at a
                lower speed.

                The solution is simply to use a powered USB hub.
                There are two solutions:
                1. Don't clip the resistor, as a faster usb bus speed is more
                important than CPU speed.
                2. If you do clip the resistor, use a powered USB hub.

                In my case, I used a powered USB hub to resolve the problem.

                cisco1:~# dd if=/dev/zero of=/tmp/x bs=4k count=100000
                100000+0 records in
                100000+0 records out
                409600000 bytes (410 MB) copied, 33.6726 seconds, 12.2 MB/s

                cisco1:~# hdparm -t -T /dev/sda

                /dev/sda:
                Timing cached reads: 96 MB in 2.03 seconds = 47.35 MB/sec
                Timing buffered disk reads: 32 MB in 3.15 seconds = 10.17 MB/sec

                I suspect the amount of slow down will vary, so I recommend even those
                who are getting reasonable speeds, try a powered hub if you want to
                get more speed.

                Bill
              • Tory Clement
                Would there be a way to jumper the clipped resistor to not lose the voltage?
                Message 7 of 18 , Mar 27, 2008
                  Would there be a way to jumper the clipped resistor to not lose the voltage?

                  On Thu, Mar 27, 2008 at 10:48 AM, docbillnet <yahoo@...> wrote:
                  >
                  >
                  >
                  >
                  >
                  >
                  > I found the cause of the problem, and the solution.
                  >
                  > When you clip the resistor to speed up the CPU, the voltage to the USB
                  > ports drops below specification. This causes devices to run at a
                  > lower speed.
                  >
                  > The solution is simply to use a powered USB hub.
                  > There are two solutions:
                  > 1. Don't clip the resistor, as a faster usb bus speed is more
                  > important than CPU speed.
                  > 2. If you do clip the resistor, use a powered USB hub.
                  >
                  > In my case, I used a powered USB hub to resolve the problem.
                  >
                  > cisco1:~# dd if=/dev/zero of=/tmp/x bs=4k count=100000
                  > 100000+0 records in
                  > 100000+0 records out
                  > 409600000 bytes (410 MB) copied, 33.6726 seconds, 12.2 MB/s
                  >
                  > cisco1:~# hdparm -t -T /dev/sda
                  >
                  > /dev/sda:
                  > Timing cached reads: 96 MB in 2.03 seconds = 47.35 MB/sec
                  > Timing buffered disk reads: 32 MB in 3.15 seconds = 10.17 MB/sec
                  >
                  > I suspect the amount of slow down will vary, so I recommend even those
                  > who are getting reasonable speeds, try a powered hub if you want to
                  > get more speed.
                  >
                  > Bill
                  >
                  >
                • Carl Lowenstein
                  ... Interesting. Surely the large drives (500GB and 1.5TB) are externally powered and do not put a power load on the USB connection. Apparently this is
                  Message 8 of 18 , Mar 27, 2008
                    On 3/27/08, docbillnet <yahoo@...> wrote:
                    >
                    > I found the cause of the problem, and the solution.
                    >
                    > When you clip the resistor to speed up the CPU, the voltage to the USB
                    > ports drops below specification. This causes devices to run at a
                    > lower speed.
                    >
                    > The solution is simply to use a powered USB hub.
                    > There are two solutions:
                    > 1. Don't clip the resistor, as a faster usb bus speed is more
                    > important than CPU speed.
                    > 2. If you do clip the resistor, use a powered USB hub.
                    >
                    > In my case, I used a powered USB hub to resolve the problem.
                    >
                    > cisco1:~# dd if=/dev/zero of=/tmp/x bs=4k count=100000
                    > 100000+0 records in
                    > 100000+0 records out
                    > 409600000 bytes (410 MB) copied, 33.6726 seconds, 12.2 MB/s
                    >
                    > cisco1:~# hdparm -t -T /dev/sda
                    >
                    > /dev/sda:
                    > Timing cached reads: 96 MB in 2.03 seconds = 47.35 MB/sec
                    > Timing buffered disk reads: 32 MB in 3.15 seconds = 10.17 MB/sec
                    >
                    > I suspect the amount of slow down will vary, so I recommend even those
                    > who are getting reasonable speeds, try a powered hub if you want to
                    > get more speed.

                    Interesting. Surely the large drives (500GB and 1.5TB) are externally
                    powered and do not put a power load on the USB connection.

                    Apparently this is something else, dropping the signal voltage. Seems
                    like a strange interaction.

                    carl
                    --
                    carl lowenstein
                    marine physical lab, u.c. san diego
                    clowenstein@...
                  • docbillnet
                    ... Yes. It was a surprising result for me as well. I do know that most USB devices will drop to 1.1 speed, when under powered. But I would think that would
                    Message 9 of 18 , Mar 27, 2008
                      --- In nslu2-general@yahoogroups.com, "Carl Lowenstein"
                      <clowenstein@...> wrote:
                      >
                      > Interesting. Surely the large drives (500GB and 1.5TB) are externally
                      > powered and do not put a power load on the USB connection.
                      >
                      > Apparently this is something else, dropping the signal voltage. Seems
                      > like a strange interaction.
                      >
                      > carl

                      Yes. It was a surprising result for me as well. I do know that most
                      USB devices will drop to 1.1 speed, when under powered. But I would
                      think that would not apply to self powered devices. My best guess is
                      even the slightest power draw causes the internal hub to drop speed.
                      When I tested with the non-powered hub, the hub itself has a light
                      that draws a small amount of power. When I tested with the memory
                      stick, the memory stick is completely powered by USB. So in both
                      cases I was drawing some power off the USB port.

                      Perhaps too, the there is a problem with the power supply that came
                      with my refurbished unit. I will have a better idea of the problem,
                      when I configure my second NSLU2 and see if it has improved
                      performance by a usb hub prior to removing the resistor to over clock.

                      Bill
                    • Mike (mwester)
                      ... This is a surprising result. Can you provide some additional detail on how you determined this? What is the voltage on the bus with the resistor, vs
                      Message 10 of 18 , Mar 27, 2008
                        docbillnet wrote:
                        > I found the cause of the problem, and the solution.
                        >
                        > When you clip the resistor to speed up the CPU, the voltage to the USB
                        > ports drops below specification. This causes devices to run at a
                        > lower speed.

                        This is a surprising result.

                        Can you provide some additional detail on how you determined this? What
                        is the voltage on the bus with the resistor, vs without? What other
                        devices are plugged into the NSLU2 (the +5V on the USB ports are all
                        directly wired together to the main +5V within the device, i.e. there is
                        no isolation between ports themselves, or ports and the device itself)?

                        Can you also check to see that your power supply is adequate? (There
                        seems to be an unusually high number of reports of defective power
                        supplies being shipped -- either that or we've just managed to track
                        more problems down to this root cause lately.)

                        Thanks,
                        Mike
                      • docbillnet
                        ... A lot of speculation, and very little evidence. Call it an educated guess, not an experimentally determined fact. The only experimentally determined
                        Message 11 of 18 , Mar 27, 2008
                          --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@...>
                          wrote:
                          >
                          > docbillnet wrote:
                          > This is a surprising result.
                          >
                          > Can you provide some additional detail on how you determined this?

                          A lot of speculation, and very little evidence. Call it an educated
                          guess, not an experimentally determined fact. The only experimentally
                          determined fact in this case us using a powered hub speeds up my USB
                          ports by a factor of 6. It is an assumption that this is because of a
                          voltage drop in the NSLU2.

                          I will try and find my multimeter and verify my guess. If the second
                          unit produces the same results, I should be able to measure voltage
                          prior to cutting the resistor. I see on the wiki someone already
                          measured voltages in their unit with and without the resister, and
                          came up with a 0.1 V difference. If I see the same 0.1 V drop, then I
                          would also need to have a bad power supply, as typically you have to
                          drop the voltage above 4.7V is sufficient for USB 2.0.

                          > is the voltage on the bus with the resistor, vs without? What other
                          > devices are plugged into the NSLU2 (the +5V on the USB ports are all
                          > directly wired together to the main +5V within the device, i.e.
                          there is
                          > no isolation between ports themselves, or ports and the device itself)?
                          >
                          > Can you also check to see that your power supply is adequate? (There
                          > seems to be an unusually high number of reports of defective power
                          > supplies being shipped -- either that or we've just managed to track
                          > more problems down to this root cause lately.)

                          Possibly. Typically, I find there is a higher chance of getting a
                          defective part in a refurbished unit. Typically they don't know why
                          the original owner returned it, so if the problem does not show up in
                          there tests, the item will be resold without fixing the actual problem.

                          Bill
                        • Mike (mwester)
                          ... Ok. So for the benefit of the community here, there exists *NO* objective evidence that removal of that resistor to de-underclock the device actually
                          Message 12 of 18 , Mar 27, 2008
                            docbillnet wrote:
                            > --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@...>
                            > wrote:
                            >> docbillnet wrote:
                            >> This is a surprising result.
                            >>
                            >> Can you provide some additional detail on how you determined this?
                            >
                            > A lot of speculation, and very little evidence. Call it an educated
                            > guess, not an experimentally determined fact. The only experimentally
                            > determined fact in this case us using a powered hub speeds up my USB
                            > ports by a factor of 6. It is an assumption that this is because of a
                            > voltage drop in the NSLU2.

                            Ok. So for the benefit of the community here, there exists *NO*
                            objective evidence that removal of that resistor to de-underclock the
                            device actually results in a voltage drop on the USB port, and further
                            there's not yet any evidence to support that any voltage drop that might
                            occur is sufficient to cause any USB problems, or cause anything to go
                            out of spec -- assuming of course, that the unit's PSU is functioning
                            correctly.

                            I just want to make that clear, lest there be a huge group of very
                            alarmed users out there suddenly wondering where they can find a new
                            resistor to solder back on, based on "a lot of speculation".

                            Let's apply the standard techniques - measure and test, and then see if
                            in fact there are any concrete statements we can say about removal of
                            that resistor and voltage at the USB and the specs.

                            Regards,
                            Mike (mwester)
                          • Rod Whitby
                            ... Agreed. We have never had any evidence of any kind that removing the underclocking resistor has any impact on nslu2 functionality, stability, longevity,
                            Message 13 of 18 , Mar 27, 2008
                              Mike (mwester) wrote:
                              > Ok. So for the benefit of the community here, there exists *NO*
                              > objective evidence that removal of that resistor to de-underclock the
                              > device actually results in a voltage drop on the USB port, and further
                              > there's not yet any evidence to support that any voltage drop that might
                              > occur is sufficient to cause any USB problems, or cause anything to go
                              > out of spec -- assuming of course, that the unit's PSU is functioning
                              > correctly.
                              >
                              > I just want to make that clear, lest there be a huge group of very
                              > alarmed users out there suddenly wondering where they can find a new
                              > resistor to solder back on, based on "a lot of speculation".

                              Agreed. We have never had any evidence of any kind that removing the
                              underclocking resistor has any impact on nslu2 functionality, stability,
                              longevity, or performance (apart from the intended doubling of speed).

                              We also know that in the latest production models of the NSLU2, a
                              similar resistor is removed (well, not added in the first place) at the
                              factory to achieve the same result. The resistors simply put a 0 or 1
                              on a reset-time sampling line which tells the IXP what speed to run at.

                              -- Rod
                            • Rod Whitby
                              ... We already know that the USB voltage supply circuitry in the NSLU2 is woeful. You should always use an externally powered hard drive with an NSLU2 - never
                              Message 14 of 18 , Mar 27, 2008
                                docbillnet wrote:
                                > --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@...>
                                > wrote:
                                >> docbillnet wrote:
                                >> This is a surprising result.
                                >>
                                >> Can you provide some additional detail on how you determined this?
                                >
                                > A lot of speculation, and very little evidence. Call it an educated
                                > guess, not an experimentally determined fact. The only experimentally
                                > determined fact in this case us using a powered hub speeds up my USB
                                > ports by a factor of 6. It is an assumption that this is because of a
                                > voltage drop in the NSLU2.

                                We already know that the USB voltage supply circuitry in the NSLU2 is
                                woeful. You should always use an externally powered hard drive with an
                                NSLU2 - never use a bus-powered hard drive (it may work 90% of the time,
                                but one day the spin-up current surge will be too much for the NSLU2's
                                power brick, and the disk won't be recognised by the slug.

                                -- Rod
                              • bloedmann999
                                ... itself)? ... Just for comparison on my Debian Slug with a single Maxtor drive connected: Direct connection without USB hub: Slug2: hdparm -t -T /dev/sda
                                Message 15 of 18 , Mar 28, 2008
                                  --- In nslu2-general@yahoogroups.com, "docbillnet" <yahoo@...> wrote:
                                  >
                                  > --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@>
                                  > wrote:
                                  > >
                                  > > docbillnet wrote:
                                  > > This is a surprising result.
                                  > >
                                  > > Can you provide some additional detail on how you determined this?
                                  >
                                  > A lot of speculation, and very little evidence. Call it an educated
                                  > guess, not an experimentally determined fact. The only experimentally
                                  > determined fact in this case us using a powered hub speeds up my USB
                                  > ports by a factor of 6. It is an assumption that this is because of a
                                  > voltage drop in the NSLU2.
                                  >
                                  > I will try and find my multimeter and verify my guess. If the second
                                  > unit produces the same results, I should be able to measure voltage
                                  > prior to cutting the resistor. I see on the wiki someone already
                                  > measured voltages in their unit with and without the resister, and
                                  > came up with a 0.1 V difference. If I see the same 0.1 V drop, then I
                                  > would also need to have a bad power supply, as typically you have to
                                  > drop the voltage above 4.7V is sufficient for USB 2.0.
                                  >
                                  > > is the voltage on the bus with the resistor, vs without? What other
                                  > > devices are plugged into the NSLU2 (the +5V on the USB ports are all
                                  > > directly wired together to the main +5V within the device, i.e.
                                  > there is
                                  > > no isolation between ports themselves, or ports and the device
                                  itself)?
                                  > >
                                  > > Can you also check to see that your power supply is adequate? (There
                                  > > seems to be an unusually high number of reports of defective power
                                  > > supplies being shipped -- either that or we've just managed to track
                                  > > more problems down to this root cause lately.)
                                  >
                                  > Possibly. Typically, I find there is a higher chance of getting a
                                  > defective part in a refurbished unit. Typically they don't know why
                                  > the original owner returned it, so if the problem does not show up in
                                  > there tests, the item will be resold without fixing the actual problem.
                                  >
                                  > Bill
                                  >
                                  Just for comparison on my Debian Slug with a single Maxtor drive
                                  connected:

                                  Direct connection without USB hub:

                                  Slug2: hdparm -t -T /dev/sda

                                  /dev/sda:
                                  Timing cached reads: 96 MB in 2.03 seconds = 47.37 MB/sec
                                  Timing buffered disk reads: 32 MB in 3.18 seconds = 10.06 MB/sec
                                  # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                  100000+0 Datensätze ein
                                  100000+0 Datensätze aus
                                  409600000 Bytes (410 MB) kopiert, 34,6773 Sekunden, 11,8 MB/s
                                  # dd if=/tmp/x of=/dev/null bs=4096
                                  100000+0 Datensätze ein
                                  100000+0 Datensätze aus
                                  409600000 Bytes (410 MB) kopiert, 31,3567 Sekunden, 13,1 MB/s

                                  Now after attachment of the disk via a USB 2 powered hub:

                                  # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                  100000+0 Datensätze ein
                                  100000+0 Datensätze aus
                                  409600000 Bytes (410 MB) kopiert, 35,9292 Sekunden, 11,4 MB/s
                                  # dd if=/tmp/x of=/dev/null bs=4096
                                  100000+0 Datensätze ein
                                  100000+0 Datensätze aus
                                  409600000 Bytes (410 MB) kopiert, 31,6457 Sekunden, 12,9 MB/s
                                  # hdparm -t -T /dev/sda

                                  /dev/sda:
                                  Timing cached reads: 96 MB in 2.03 seconds = 47.26 MB/sec
                                  Timing buffered disk reads: 34 MB in 3.12 seconds = 10.90 MB/sec

                                  My Debian Slug has the resistor cut.
                                  I'll see if I can carry out the same comparison on my older Unslung
                                  slug too.

                                  Cheers Brian
                                • bloedmann999
                                  ... other ... all ... (There ... track ... problem. ... And now without a hub, to a 320GB WD USB-2 Drive on Unslung: Welcome to Unslung
                                  Message 16 of 18 , Mar 28, 2008
                                    --- In nslu2-general@yahoogroups.com, "bloedmann999"
                                    <Brian_Dorling@...> wrote:
                                    >
                                    > --- In nslu2-general@yahoogroups.com, "docbillnet" <yahoo@> wrote:
                                    > >
                                    > > --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@>
                                    > > wrote:
                                    > > >
                                    > > > docbillnet wrote:
                                    > > > This is a surprising result.
                                    > > >
                                    > > > Can you provide some additional detail on how you determined this?
                                    > >
                                    > > A lot of speculation, and very little evidence. Call it an educated
                                    > > guess, not an experimentally determined fact. The only experimentally
                                    > > determined fact in this case us using a powered hub speeds up my USB
                                    > > ports by a factor of 6. It is an assumption that this is because of a
                                    > > voltage drop in the NSLU2.
                                    > >
                                    > > I will try and find my multimeter and verify my guess. If the second
                                    > > unit produces the same results, I should be able to measure voltage
                                    > > prior to cutting the resistor. I see on the wiki someone already
                                    > > measured voltages in their unit with and without the resister, and
                                    > > came up with a 0.1 V difference. If I see the same 0.1 V drop, then I
                                    > > would also need to have a bad power supply, as typically you have to
                                    > > drop the voltage above 4.7V is sufficient for USB 2.0.
                                    > >
                                    > > > is the voltage on the bus with the resistor, vs without? What
                                    other
                                    > > > devices are plugged into the NSLU2 (the +5V on the USB ports are
                                    all
                                    > > > directly wired together to the main +5V within the device, i.e.
                                    > > there is
                                    > > > no isolation between ports themselves, or ports and the device
                                    > itself)?
                                    > > >
                                    > > > Can you also check to see that your power supply is adequate?
                                    (There
                                    > > > seems to be an unusually high number of reports of defective power
                                    > > > supplies being shipped -- either that or we've just managed to
                                    track
                                    > > > more problems down to this root cause lately.)
                                    > >
                                    > > Possibly. Typically, I find there is a higher chance of getting a
                                    > > defective part in a refurbished unit. Typically they don't know why
                                    > > the original owner returned it, so if the problem does not show up in
                                    > > there tests, the item will be resold without fixing the actual
                                    problem.
                                    > >
                                    > > Bill
                                    > >
                                    > Just for comparison on my Debian Slug with a single Maxtor drive
                                    > connected:
                                    >
                                    > Direct connection without USB hub:
                                    >
                                    > Slug2: hdparm -t -T /dev/sda
                                    >
                                    > /dev/sda:
                                    > Timing cached reads: 96 MB in 2.03 seconds = 47.37 MB/sec
                                    > Timing buffered disk reads: 32 MB in 3.18 seconds = 10.06 MB/sec
                                    > # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                    > 100000+0 Datensätze ein
                                    > 100000+0 Datensätze aus
                                    > 409600000 Bytes (410 MB) kopiert, 34,6773 Sekunden, 11,8 MB/s
                                    > # dd if=/tmp/x of=/dev/null bs=4096
                                    > 100000+0 Datensätze ein
                                    > 100000+0 Datensätze aus
                                    > 409600000 Bytes (410 MB) kopiert, 31,3567 Sekunden, 13,1 MB/s
                                    >
                                    > Now after attachment of the disk via a USB 2 powered hub:
                                    >
                                    > # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                    > 100000+0 Datensätze ein
                                    > 100000+0 Datensätze aus
                                    > 409600000 Bytes (410 MB) kopiert, 35,9292 Sekunden, 11,4 MB/s
                                    > # dd if=/tmp/x of=/dev/null bs=4096
                                    > 100000+0 Datensätze ein
                                    > 100000+0 Datensätze aus
                                    > 409600000 Bytes (410 MB) kopiert, 31,6457 Sekunden, 12,9 MB/s
                                    > # hdparm -t -T /dev/sda
                                    >
                                    > /dev/sda:
                                    > Timing cached reads: 96 MB in 2.03 seconds = 47.26 MB/sec
                                    > Timing buffered disk reads: 34 MB in 3.12 seconds = 10.90 MB/sec
                                    >
                                    > My Debian Slug has the resistor cut.
                                    > I'll see if I can carry out the same comparison on my older Unslung
                                    > slug too.
                                    >
                                    > Cheers Brian
                                    >
                                    And now without a hub, to a 320GB WD USB-2 Drive on Unslung:

                                    Welcome to Unslung V2.3R63-uNSLUng-6.8-beta
                                    Slug1: uname -r
                                    2.4.22-xfs

                                    Slug1: hdparm -t -T /dev/sdb

                                    /dev/sdb:
                                    Timing cached reads: 82 MB in 2.02 seconds = 40.59 MB/sec
                                    Timing buffered disk reads: 26 MB in 3.13 seconds = 8.31 MB/sec


                                    Slug1: date; dd if=/dev/zero of=/tmp/x bs=4096 count=100000; date
                                    Fri Mar 28 18:29:50 CET 2008
                                    100000+0 records in
                                    100000+0 records out
                                    Fri Mar 28 18:30:43 CET 2008

                                    = 7.74 MByte/Sec

                                    Slug1: date; dd if=/tmp/x of=/dev/null bs=4096; date

                                    Fri Mar 28 18:31:30 CET 2008
                                    100000+0 records in
                                    100000+0 records out
                                    Fri Mar 28 18:32:07 CET 2008

                                    = 11.08 MByte Sec

                                    I am not going to take the powered hub down and try with that as I
                                    think the speeds are OK. I seem to remember speeds of about 3MB/sec
                                    when writing to the disk via ethernet as being pretty normal. I think
                                    these results just show what can be achieved via the USB port of the
                                    Slug. Not sure how USB-USB transfers between the two Slug USB ports
                                    would compare.

                                    Cheers Brian
                                  • bloedmann999
                                    ... this? ... educated ... experimentally ... because of a ... second ... then I ... power ... up in ... And the rest on Debian: SDA1: # dd if=/dev/zero
                                    Message 17 of 18 , Mar 28, 2008
                                      --- In nslu2-general@yahoogroups.com, "bloedmann999"
                                      <Brian_Dorling@...> wrote:
                                      >
                                      > --- In nslu2-general@yahoogroups.com, "bloedmann999"
                                      > <Brian_Dorling@> wrote:
                                      > >
                                      > > --- In nslu2-general@yahoogroups.com, "docbillnet" <yahoo@> wrote:
                                      > > >
                                      > > > --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@>
                                      > > > wrote:
                                      > > > >
                                      > > > > docbillnet wrote:
                                      > > > > This is a surprising result.
                                      > > > >
                                      > > > > Can you provide some additional detail on how you determined
                                      this?
                                      > > >
                                      > > > A lot of speculation, and very little evidence. Call it an
                                      educated
                                      > > > guess, not an experimentally determined fact. The only
                                      experimentally
                                      > > > determined fact in this case us using a powered hub speeds up my USB
                                      > > > ports by a factor of 6. It is an assumption that this is
                                      because of a
                                      > > > voltage drop in the NSLU2.
                                      > > >
                                      > > > I will try and find my multimeter and verify my guess. If the
                                      second
                                      > > > unit produces the same results, I should be able to measure voltage
                                      > > > prior to cutting the resistor. I see on the wiki someone already
                                      > > > measured voltages in their unit with and without the resister, and
                                      > > > came up with a 0.1 V difference. If I see the same 0.1 V drop,
                                      then I
                                      > > > would also need to have a bad power supply, as typically you have to
                                      > > > drop the voltage above 4.7V is sufficient for USB 2.0.
                                      > > >
                                      > > > > is the voltage on the bus with the resistor, vs without? What
                                      > other
                                      > > > > devices are plugged into the NSLU2 (the +5V on the USB ports are
                                      > all
                                      > > > > directly wired together to the main +5V within the device, i.e.
                                      > > > there is
                                      > > > > no isolation between ports themselves, or ports and the device
                                      > > itself)?
                                      > > > >
                                      > > > > Can you also check to see that your power supply is adequate?
                                      > (There
                                      > > > > seems to be an unusually high number of reports of defective
                                      power
                                      > > > > supplies being shipped -- either that or we've just managed to
                                      > track
                                      > > > > more problems down to this root cause lately.)
                                      > > >
                                      > > > Possibly. Typically, I find there is a higher chance of getting a
                                      > > > defective part in a refurbished unit. Typically they don't know why
                                      > > > the original owner returned it, so if the problem does not show
                                      up in
                                      > > > there tests, the item will be resold without fixing the actual
                                      > problem.
                                      > > >
                                      > > > Bill
                                      > > >
                                      > > Just for comparison on my Debian Slug with a single Maxtor drive
                                      > > connected:
                                      > >
                                      > > Direct connection without USB hub:
                                      > >
                                      > > Slug2: hdparm -t -T /dev/sda
                                      > >
                                      > > /dev/sda:
                                      > > Timing cached reads: 96 MB in 2.03 seconds = 47.37 MB/sec
                                      > > Timing buffered disk reads: 32 MB in 3.18 seconds = 10.06 MB/sec
                                      > > # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                      > > 100000+0 Datensätze ein
                                      > > 100000+0 Datensätze aus
                                      > > 409600000 Bytes (410 MB) kopiert, 34,6773 Sekunden, 11,8 MB/s
                                      > > # dd if=/tmp/x of=/dev/null bs=4096
                                      > > 100000+0 Datensätze ein
                                      > > 100000+0 Datensätze aus
                                      > > 409600000 Bytes (410 MB) kopiert, 31,3567 Sekunden, 13,1 MB/s
                                      > >
                                      > > Now after attachment of the disk via a USB 2 powered hub:
                                      > >
                                      > > # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                      > > 100000+0 Datensätze ein
                                      > > 100000+0 Datensätze aus
                                      > > 409600000 Bytes (410 MB) kopiert, 35,9292 Sekunden, 11,4 MB/s
                                      > > # dd if=/tmp/x of=/dev/null bs=4096
                                      > > 100000+0 Datensätze ein
                                      > > 100000+0 Datensätze aus
                                      > > 409600000 Bytes (410 MB) kopiert, 31,6457 Sekunden, 12,9 MB/s
                                      > > # hdparm -t -T /dev/sda
                                      > >
                                      > > /dev/sda:
                                      > > Timing cached reads: 96 MB in 2.03 seconds = 47.26 MB/sec
                                      > > Timing buffered disk reads: 34 MB in 3.12 seconds = 10.90 MB/sec
                                      > >
                                      > > My Debian Slug has the resistor cut.
                                      > > I'll see if I can carry out the same comparison on my older Unslung
                                      > > slug too.
                                      > >
                                      > > Cheers Brian
                                      > >
                                      > And now without a hub, to a 320GB WD USB-2 Drive on Unslung:
                                      >
                                      > Welcome to Unslung V2.3R63-uNSLUng-6.8-beta
                                      > Slug1: uname -r
                                      > 2.4.22-xfs
                                      >
                                      > Slug1: hdparm -t -T /dev/sdb
                                      >
                                      > /dev/sdb:
                                      > Timing cached reads: 82 MB in 2.02 seconds = 40.59 MB/sec
                                      > Timing buffered disk reads: 26 MB in 3.13 seconds = 8.31 MB/sec
                                      >
                                      >
                                      > Slug1: date; dd if=/dev/zero of=/tmp/x bs=4096 count=100000; date
                                      > Fri Mar 28 18:29:50 CET 2008
                                      > 100000+0 records in
                                      > 100000+0 records out
                                      > Fri Mar 28 18:30:43 CET 2008
                                      >
                                      > = 7.74 MByte/Sec
                                      >
                                      > Slug1: date; dd if=/tmp/x of=/dev/null bs=4096; date
                                      >
                                      > Fri Mar 28 18:31:30 CET 2008
                                      > 100000+0 records in
                                      > 100000+0 records out
                                      > Fri Mar 28 18:32:07 CET 2008
                                      >
                                      > = 11.08 MByte Sec
                                      >
                                      > I am not going to take the powered hub down and try with that as I
                                      > think the speeds are OK. I seem to remember speeds of about 3MB/sec
                                      > when writing to the disk via ethernet as being pretty normal. I think
                                      > these results just show what can be achieved via the USB port of the
                                      > Slug. Not sure how USB-USB transfers between the two Slug USB ports
                                      > would compare.
                                      >
                                      > Cheers Brian
                                      >
                                      And the rest on Debian:

                                      SDA1:
                                      # dd if=/dev/zero of=/tmp/x bs=4096 count=100000
                                      100000+0 Datensätze ein
                                      100000+0 Datensätze aus
                                      409600000 Bytes (410 MB) kopiert, 35,2087 Sekunden, 11,6 MB/s

                                      SDB1:
                                      # dd if=/dev/zero of=/mnt/sdb1/tmp/x bs=4096 count=100000
                                      100000+0 Datensätze ein
                                      100000+0 Datensätze aus
                                      409600000 Bytes (410 MB) kopiert, 34,834 Sekunden, 11,8 MB/s


                                      Now between the two Drives:
                                      # dd if=/tmp/x of=/mnt/sdb1/tmp/x bs=4096
                                      100000+0 Datensätze ein
                                      100000+0 Datensätze aus
                                      409600000 Bytes (410 MB) kopiert, 56,4031 Sekunden, 7,3 MB/s

                                      Reverse direction:
                                      # dd if=/mnt/sdb1/tmp/x of=/tmp/x bs=4096
                                      100000+0 Datensätze ein
                                      100000+0 Datensätze aus
                                      409600000 Bytes (410 MB) kopiert, 56,7983 Sekunden, 7,2 MB/s

                                      But, transferring via rsync a WinXP partimage file between the two USB
                                      drives using --progress I am getting around 2.9MByte/sec being shown.

                                      Results:
                                      winXP-2.000
                                      1801840905 100% 2.89MB/s 0:09:53

                                      sent 1802061194 bytes received 48 bytes 3031221.60 bytes/sec
                                      total size is 6489685925 speedup is 3.60

                                      real 9m54.255s
                                      user 4m40.030s
                                      sys 4m18.910s

                                      Which gives me 3.034 MByte/Sec effective. On this xfer the CPU is
                                      pegged at 100%. Starting HTOP reduces the xfer rate by about
                                      200KByte/Sec.



                                      On an rsync of a directory with lots of small files I had:

                                      Total bytes sent: 91576843
                                      Total bytes received: 111648

                                      sent 91576843 bytes received 111648 bytes 970248.58 bytes/sec
                                      total size is 93956026 speedup is 1.02

                                      real 1m34.507s
                                      user 0m21.840s
                                      sys 0m47.210s

                                      Which puts the effective xfer rate under 1 MByte/Sec.

                                      Cheers Brian
                                    • cnczane
                                      I m sorry I don t buy it. I regularly get 3.0+MB/s (quoting rsync s --progress) between slugs with two hard-drives each. All four of my slugs are
                                      Message 18 of 18 , Jan 20, 2010
                                        I'm sorry I don't buy it.

                                        I regularly get 3.0+MB/s (quoting rsync's --progress) between slugs with two hard-drives each. All four of my slugs are de-underclocked, and all of their hard drives are externally powered. I started with Maxtors but lately use internal harddrives upended in BlacX.

                                        I did have compression enabled once, because I am silly, and the speed dropped into the KB-ish range. (Compressing compressed files only expands time...)

                                        Cheers!
                                        --
                                        CZ

                                        --- In nslu2-general@yahoogroups.com, "Mike (mwester)" <mwester@...> wrote:
                                        >
                                        > docbillnet wrote:
                                        > > Hello,
                                        > >
                                        > > I just purchased a couple of NSLU2 from tigerdirect.ca, and I have
                                        > > followed the instructions for installing Linux on one. Overall, it
                                        > > seems to be working well, especially since I cut the resistor.
                                        > > However, I wanted to backup data from three 500 GB drives to a 1.5 TB
                                        > > drive. However, when I used the --progress option with rsync, it
                                        > > reported the transfer speed is 2.0 MB/s. After a quick back of the
                                        > > envelope calculation, I concluded it will take 8 days to copy my data
                                        > > from the three drives.
                                        > >
                                        > > So, I shutdown the NSLU2, and plugged the same drives and USB hub into
                                        > > my laptop. I issued the same rsync command, and my transfer rate is
                                        > > 20 MB/s.
                                        > >
                                        > > Is this a typical transfer speed with an NSLU2? If not, how do I go
                                        > > about determining the problem and speeding up the transfer? From what
                                        > > I've seen in this forum, I would expect a transfer speed between
                                        > > drives of about 6 MB/s, but I can't be certain since the messages were
                                        > > all related to how long it took to transfer files between a usb drive
                                        > > and the network. Perhaps, the USB bus is much slower when it is both
                                        > > reading and writing at the same time. Or perhaps rsync has too much
                                        > > overhead, and I need to use a different program to copy data.
                                        > >
                                        > > Bill
                                        >
                                        > What firmware did you install?
                                        > How are the drives mounted (can you provide the output from the mount
                                        > command)?
                                        > What exactly is the rsync command you issued?
                                        >
                                        > Mike (mwester)
                                        >
                                      Your message has been successfully submitted and would be delivered to recipients shortly.