Loading ...
Sorry, an error occurred while loading the content.

Maximizing Slug Fileserver Speed

Expand Messages
  • Brian Wood
    Trying to see how close I can get to that elusive 80Mbs , I really want to get as much performance as I can out of the slug as a fileserver. The slug will
    Message 1 of 6 , Jul 1, 2006
    • 0 Attachment
      Trying to see how close I can get to that elusive "80Mbs", I really
      want to get as much performance as I can out of the slug as a
      fileserver.

      The slug will serve MPEG-2 video files, mostly over 1GB in size.

      I'm making a couple of assumptions, please let me know if you think
      they are wrong:

      I should use NFS and not Samba for maximum speed (not serving to
      Windows).

      I should use a Big-Endian OS (not sure how much difference this might
      make).

      I should De-underclock the slug (seems obvious).

      OK, if I'm wrong let me know, now a couple of questions:

      Should I use a hard drive or a flash device for the OS ? (I'm
      assuming NFS would not give optimum speed).

      What filesystem should I use ?

      Is there a particular USB/IDE chipset(s) that gives the best
      performance?

      Are there any user settable TCP/IP settings that might help optimize
      speed.

      Are there any filesystem options that might help optimize speed ?

      Any other configuration settings that might help optimize speed ?

      Obviously there may be no hard and fast answers to the above, but I'd
      like to hear any opinions, discoveries, ideas etc.

      Much thanks for any info.


      Brian Wood
      beww@...
    • Attila Csipa
      ... What nobody seems to mention is the importance of the type of disks they are using. 2.5 disks are not just the same but smaller than 3.5 disks. If you
      Message 2 of 6 , Jul 2, 2006
      • 0 Attachment
        On Sunday 02 July 2006 05:11, Brian Wood wrote:
        > Is there a particular USB/IDE chipset(s) that gives the best
        > performance?

        What nobody seems to mention is the importance of the type of disks they are
        using. 2.5" disks are not 'just the same but smaller' than 3.5" disks. If you
        want to transfer large files, you will get noticeably lower speeds since
        built-in disk cache won't help you and the lower spin rate of 2.5" disks
        really comes out in those cases (all this assuming that the USB2IDE chipset
        can actually sustain that throughput). They are generally designed for
        notebooks (small size, lower power, more resistance to shock and heat, not
        top performance).
      • Yann E. MORIN
        Brian, All, ... How many at the same time? Which bitrate: 4Mibps (SDTV) or 8Mibps (HDTV)? What s between your slug and the player: WiFi, cat-5, PLT? One or
        Message 3 of 6 , Jul 2, 2006
        • 0 Attachment
          Brian,
          All,

          On Sunday 02 July 2006 051, Brian Wood wrote:
          > The slug will serve MPEG-2 video files, mostly over 1GB in size.

          How many at the same time? Which bitrate: 4Mibps (SDTV) or >8Mibps (HDTV)?
          What's between your slug and the player: WiFi, cat-5, PLT? One or more hub
          or switch? (see at bottom of mail).

          > I should use NFS and not Samba for maximum speed (not serving to
          > Windows).

          To serve video (or music as well), why don't you use streaming?
          That would have the benefit of adding no file system overhead on
          the network, thus maximizing the bandwidth for the actual data
          (provided your storage medium can sustain the throughput).

          > I should use a Big-Endian OS (not sure how much difference this might
          > make).

          From my experiments, that's not noticeable by timing with hardware,
          and even less noticeable for a human being. That said, if you need
          horsepower at the same time you transfer data, then you may loose
          a few ticks, but that's nothing compared to any context-switch.

          > I should De-underclock the slug (seems obvious).

          That should help, but again, serving video via streaming is mostly
          copy-from-disk-to-memory-to-network, and your limiting factors will most
          probably be (in path-order):
          - disk-throughtput,
          - IDE2USB adapter,
          - USB throughput (you should be fine here!),
          - memory latency,
          - network throughput,
          rather than CPU performance

          > Should I use a hard drive or a flash device for the OS ? (I'm
          > assuming NFS would not give optimum speed).

          I don't get it right. Were your planing on an NFS-root? That definitely
          is a bad idea to maximise your bandwidth usage!
          Anyway a hard drive for the rootfs would be better, you could even add
          some swap to help in those moments where you'd need it.

          > What filesystem should I use ?

          To store files, any _decent_ FS should do. ext2/3 or even reiserfs should be
          fine. Avoid FAT & co. and NTFS. Avoid any compressed FS (cramfs, squashfs,
          JFFS2...) because those _are_ CPU-bound.

          To share files, don't use any file system. Use streaming. That's faster.
          If you definitely need a file system, then NFS is the best choice over samba.

          > Are there any user settable TCP/IP settings that might help optimize
          > speed.

          NFS, try rsize=16384 and wsize=16384 (or even 32768, but not bigger).
          man 5 nfs

          > Are there any filesystem options that might help optimize speed ?

          On your storage file system, use noatime (for those file systems supporting
          atime). man 8 mount

          > Any other configuration settings that might help optimize speed ?

          Add RAM to your slug to avoid swapping (although file-serving should not
          cause swapping).

          > Obviously there may be no hard and fast answers to the above, but I'd
          > like to hear any opinions, discoveries, ideas etc.

          I would stream media data (video/music), and serve other data via NFS. Thus
          I could apply some QoS on the media streams to ensure they get delivered
          reliably (in terms of latency and throughput) while my other services gets
          slowed in favor of media-streams.

          > Much thanks for any info.

          You're my guest. Anyway, I'm slowly aiming at the same kind of setup, but
          much work is before me.. :-)

          Regards,
          Yann E. MORIN.

          Note from the top:
          My setup is (not counting the USB hub and DSL in the router):

          HDD <-+ nslu2 PC: br0 router: br0
          HDD <-+ / \ / \ / \
          HDD <-+-> usb eth0 <-> eth1 eth0 <-> switch <-> PLT <-> eth0 wifi <-> PC
          \ | \
          \-> PC PC \-> PC
          +-> PC +-> PC

          ( A bit of overhead, he? :-] ) Thus I'm limited to the narower tube in the path,
          that is PLT (about 6Mibps). And I'm using a bridge in the first PC because it
          is the one I use most frequently, and which needs the faster access to the slug.
          YEM.

          --
          .-----------------.--------------------.------------------.--------------------.
          | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
          | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^ |
          | --==< °_° >==-- °---.----------------: X AGAINST | /e\ There is no |
          | web: ymorin.free.fr | SETI@home 3808 | / \ HTML MAIL | """ conspiracy. |
          °---------------------°----------------°------------------°--------------------°
        • Brian Wood
          ... Good point. What you say is absolutely correct for drives being used with their native (usually ATA) interface (although we are starting to see some 7200
          Message 4 of 6 , Jul 2, 2006
          • 0 Attachment
            On Jul 2, 2006, at 3:28 AM, Attila Csipa wrote:

            > On Sunday 02 July 2006 05:11, Brian Wood wrote:
            >> Is there a particular USB/IDE chipset(s) that gives the best
            >> performance?
            >
            > What nobody seems to mention is the importance of the type of disks
            > they are
            > using. 2.5" disks are not 'just the same but smaller' than 3.5"
            > disks. If you
            > want to transfer large files, you will get noticeably lower speeds
            > since
            > built-in disk cache won't help you and the lower spin rate of 2.5"
            > disks
            > really comes out in those cases (all this assuming that the USB2IDE
            > chipset
            > can actually sustain that throughput). They are generally designed for
            > notebooks (small size, lower power, more resistance to shock and
            > heat, not
            > top performance).

            Good point. What you say is absolutely correct for drives being used
            with their native (usually ATA) interface (although we are starting
            to see some 7200 RPM 2.5" drives. I've also seen some 2GB and 4GB all
            solid-state 2.5" drive replacements).

            I guess I figured that the performance loss in going to the USB
            interface was so great that it really didn't matter how fast the
            drive was to start with.

            Now I'm not so sure. I guess I'll just have to try using different
            drives and see if there is a difference.
          • Brian Wood
            ... These are SD files recorded by a MythTV system using PVR capture cards. They are about 2.2GB per hour (more or less full D1). The network is 100-base-T
            Message 5 of 6 , Jul 2, 2006
            • 0 Attachment
              On Jul 2, 2006, at 6:25 AM, Yann E. MORIN wrote:

              > Brian,
              > All,
              >
              > On Sunday 02 July 2006 051, Brian Wood wrote:
              >> The slug will serve MPEG-2 video files, mostly over 1GB in size.
              >
              > How many at the same time? Which bitrate: 4Mibps (SDTV) or >8Mibps
              > (HDTV)?
              > What's between your slug and the player: WiFi, cat-5, PLT? One or
              > more hub
              > or switch? (see at bottom of mail).

              These are SD files recorded by a MythTV system using PVR capture
              cards. They are about 2.2GB per hour (more or less full D1). The
              network is 100-base-T hard-wired (CAT-5E) with one switch, not
              including the switch in the router.


              >
              >> I should use NFS and not Samba for maximum speed (not serving to
              >> Windows).
              >
              > To serve video (or music as well), why don't you use streaming?
              > That would have the benefit of adding no file system overhead on
              > the network, thus maximizing the bandwidth for the actual data
              > (provided your storage medium can sustain the throughput).

              I should have said that my purpose is not to "play" the files, but to
              give multiple machines access to the files in order edit them and
              create DVD images to burn.

              As such I don't think streaming would work, certainly not without
              major changes to the editing software. Throughput is not an issue the
              way it would be if playing, as there is no "real time" to keep up
              with, but obviously the faster the server the less time spent waiting
              while editing.


              >
              >> I should use a Big-Endian OS (not sure how much difference this might
              >> make).
              >
              >> From my experiments, that's not noticeable by timing with hardware,
              > and even less noticeable for a human being. That said, if you need
              > horsepower at the same time you transfer data, then you may loose
              > a few ticks, but that's nothing compared to any context-switch.

              There had been a few posts here recently that seemed to indicate that
              a TCP/IP-intensive machine would be more efficient if running BE, but
              I suspected the difference would be minor.

              But I'm tending to run the BE OpenSlug in any case, certainly I don't
              need the vast package library of the full Debian release just to run
              an NFS server.

              Unless, of course, somebody could convince me that OpenSlug is not
              the best answer here.


              >
              >> I should De-underclock the slug (seems obvious).
              >
              > That should help, but again, serving video via streaming is mostly
              > copy-from-disk-to-memory-to-network, and your limiting factors will
              > most
              > probably be (in path-order):
              > - disk-throughtput,
              > - IDE2USB adapter,
              > - USB throughput (you should be fine here!),
              > - memory latency,
              > - network throughput,
              > rather than CPU performance



              >
              >> Should I use a hard drive or a flash device for the OS ? (I'm
              >> assuming NFS would not give optimum speed).
              >
              > I don't get it right. Were your planing on an NFS-root? That
              > definitely
              > is a bad idea to maximise your bandwidth usage!
              > Anyway a hard drive for the rootfs would be better, you could even add
              > some swap to help in those moments where you'd need it.

              No NFS root, that would definitely not give me optimum speed. Just
              deciding between a hard drive and a flash device for root. I've never
              tried a flash stick, always used drives with nice large swap partitions.

              I've seen some "Fast Flash" NAND devices that are drop-in
              replacements for 2.5" hard drives. They claim 17.7MB/sec. read/write
              burst capability and 12MB sustained rates. Expensive, but look
              awfully nice for a root filesystem, and plenty of room for swap.
              Built-in "dynamic wear leveling", which I take to mean spreading out
              the load so you don't wear out specific locations quickly.


              >
              >> What filesystem should I use ?
              >
              > To store files, any _decent_ FS should do. ext2/3 or even reiserfs
              > should be
              > fine. Avoid FAT & co. and NTFS. Avoid any compressed FS (cramfs,
              > squashfs,
              > JFFS2...) because those _are_ CPU-bound.

              I think the first 3 you mentioned are the only simple choices for a
              slug in any case. I know the Myth people go on endlessly about what
              filesystem is best, usually with very conflicting results.


              Anyway, much thanks for your input, it is appreciated.
            • Brian Wood
              Apologies for not having found this earlier: http://www.nslu2-linux.org/wiki/Info/Performance Especially: Some tests done with the copy of a 200MB file between
              Message 6 of 6 , Jul 2, 2006
              • 0 Attachment
                Apologies for not having found this earlier:

                http://www.nslu2-linux.org/wiki/Info/Performance

                Especially:

                Some tests done with the copy of a 200MB file between a NSLU2 with
                2.12-beta firmware and a Linux 2.8 (ubuntu), the test was performed
                with the filesystem "mounted" and then a simple read/write of the
                200MB via a python script.

                read write
                nfs: 5.7MB/s 2.6MB/s
                cifs: 3.5MB/s 1.9MB/s
                samba: 2.2MB/s 1.85MB/s

                nfs-server Version: 2.2beta47-2

                --titoo

                Looks like we're all doing a bit better than that, but that may also
                be with a 133Mhz. slug.
              Your message has been successfully submitted and would be delivered to recipients shortly.