Loading ...
Sorry, an error occurred while loading the content.

Re: SlugOS/BE 4.8-beta large file corruption

Expand Messages
  • Jon
    ... found ... Thanks for the replies Mike & Thomas. I remember your conversations last month & decided they didn t apply to me because they were with 4.9 &
    Message 1 of 7 , Mar 4, 2008
    • 0 Attachment
      --- In nslu2-linux@yahoogroups.com, "Mike (mwester)" <mwester@...> wrote:
      >
      > I too would be very interested in any more information that can be
      found
      > about these situations. I've been quite unable to duplicate Thomas'
      > issue with SlugOS 4.9 to date, so there must be something else that is
      > triggering this problem.
      > Mike (mwester)

      Thanks for the replies Mike & Thomas. I remember your conversations
      last month & decided they didn't apply to me because they were with
      4.9 & 4.10 using different ethernet drivers. Maybe I'm wrong!

      Anyway, I'm not seeing anything new appearing in the output of dmesg.
      I also don't get any errors or failures reported to the terminal.

      The crazy thing is that one of my tests is done copying between two
      drives both directly connected to the same slug - which seems to count
      the network drivers out of it. And still I see md5 differences.

      Another point from testing I've done since my earlier post...
      I copied 1GB of MP3 files across (via the PC so
      slug1->Samba->PC->Samba->slug2) and ran md5 on the resulting 185 files
      and found 1 difference. So my theory that it only affected large files
      is incorrect - it's just more apparent in large files because there's
      more data to look at.

      If there's anything I can do to help you pin this down Mike, then just
      shout.

      Jon
    • Drew Gibson
      ... I m not experiencing your issues but might I suggest a couple of things to try? Have you tried copying with SSH? It will checksum the data as it goes. If
      Message 2 of 7 , Mar 5, 2008
      • 0 Attachment

        Jon wrote:
        --- In nslu2-linux@yahoogroups.com, "Mike (mwester)" <mwester@...> wrote:
          
        I too would be very interested in any more information that can be
            
        found 
          
        about these situations.  I've been quite unable to duplicate Thomas' 
        issue with SlugOS 4.9 to date, so there must be something else that is 
        triggering this problem.
        Mike (mwester)
            
        Thanks for the replies Mike & Thomas. I remember your conversations
        last month & decided they didn't apply to me because they were with
        4.9 & 4.10 using different ethernet drivers. Maybe I'm wrong!
        
        Anyway, I'm not seeing anything new appearing in the output of dmesg.
        I also don't get any errors or failures reported to the terminal.
        
        The crazy thing is that one of my tests is done copying between two
        drives both directly connected to the same slug - which seems to count
        the network drivers out of it. And still I see md5 differences.
        
        Another point from testing I've done since my earlier post...
        I copied 1GB of MP3 files across (via the PC so
        slug1->Samba->PC->Samba->slug2) and ran md5 on the resulting 185 files
        and found 1 difference. So my theory that it only affected large files
        is incorrect - it's just more apparent in large files because there's
        more data to look at.
        
        If there's anything I can do to help you pin this down Mike, then just
        shout.
        
        Jon
        
        
          

        I'm not experiencing your issues but might I suggest a couple of things to try?

        Have you tried copying with SSH? It will checksum the data as it goes. If the copy goes OK but the resultant files still have an MD5SUM mismatch, then you should look at the drives. If the copies take forever, then it may be doing many retries on the transfer, so look at the network, physical and drivers.

        The ethernet driver in SlugOS maybe suspect (as per Thomas), do you have a USB ethernet dongle you could try?

        regards,

        Drew


      • Thomas Reitmayr
        Hi, I noticed that the network traffic stopped from time to time for 0.5 to 2 seconds when the errors happened. For NFS write transfers (PC- Slug) the NFS
        Message 3 of 7 , Mar 5, 2008
        • 0 Attachment
          Hi,
          I noticed that the network traffic stopped from time to time for 0.5 to 2 seconds when the errors happened. For NFS write transfers (PC->Slug) the NFS syslog entries occurred during the transfer while the PC reported an error at the end of the file transfer.
          I will try SSH as you suggested and will report the results. Unfortunately I have no USB ethernet dongle at hand.
          Thanks,
          -Thomas

          ----- Ursprüngliche Mail ----
          Von: Drew Gibson <aggibson@...>
          An: nslu2-linux@yahoogroups.com
          Gesendet: Mittwoch, den 5. März 2008, 16:03:47 Uhr
          Betreff: Re: [nslu2-linux] Re: SlugOS/BE 4.8-beta large file corruption



          I'm not experiencing your issues but might I suggest a couple of things to try?

          Have you tried copying with SSH? It will checksum the data as it goes. If the copy goes OK but the resultant files still have an MD5SUM mismatch, then you should look at the drives. If the copies take forever, then it may be doing many retries on the transfer, so look at the network, physical and drivers.

          The ethernet driver in SlugOS maybe suspect (as per Thomas), do you have a USB ethernet dongle you could try?

          regards,

          Drew





          Beginnen Sie den Tag mit den neuesten Nachrichten. Machen Sie Yahoo! zu Ihrer Startseite!
        • Jon
          ... have ... Hi Drew, thanks for the ideas. Good idea about the USB ethernet dongle. Unfortunately I don t have a USB dongle around to test it with. Unless
          Message 4 of 7 , Mar 5, 2008
          • 0 Attachment
            --- In nslu2-linux@yahoogroups.com, Drew Gibson <aggibson@...> wrote:
            > I'm not experiencing your issues but might I suggest a couple of things
            > to try?
            > Have you tried copying with SSH? It will checksum the data as it goes.
            > If the copy goes OK but the resultant files still have an MD5SUM
            > mismatch, then you should look at the drives. If the copies take
            > forever, then it may be doing many retries on the transfer, so look at
            > the network, physical and drivers.
            >
            > The ethernet driver in SlugOS maybe suspect (as per Thomas), do you
            have
            > a USB ethernet dongle you could try?

            Hi Drew,
            thanks for the ideas.

            Good idea about the USB ethernet dongle. Unfortunately I don't have a
            USB dongle around to test it with. Unless SlugOS supports USB WiFi -
            and from the problems I've had in the "conventional" Linux world with
            my Linksys adapter, I don't think that's somewhere I want to go :-)

            I ran more tests overnight... ran multiple md5sum passes on
            directories containing several GB of files. And I got varying results.
            This was with me logged into the SlugOS slug and I got differences
            checksuming files on the 500GB drive that lives on that slug,
            differences checksuming files on the 300GB drive that normally lives
            on the uNSLUng slug and also differences checksuming files that I
            copied onto a portion of the USB flash drive that contains SlugOS. So
            it would appear that the physical drive is not the factor.

            I tried a copy using scp. I copied the same 350MB file twice from the
            uNSLUng slug, pushing it to the flash drive on the SlugOS slug. Then I
            md5'ed each copy five times. The first copy gave the right checksum 2
            out of 5 times, the second copy gave the right checksum 3 out of 5
            times. A copy I did through a plain nfs copy gave variable answers on
            each md5sum but the correct answer never appeared and there would
            usually be a wrong answer that appeared more than once. My guess is
            that the repeated answer is the actual MD5 of the copied file, so the
            file copied with a straight copy is wrong, but the file copied with
            scp is correct - it's just that md5sum is having a hard time telling it.

            I did for a while wonder if the SlugOS build of md5sum might be
            faulty! So I copied the same file over Samba to a Windows PC four
            times & ran two different md5 programs on it there. Three of those
            copies gave consistent results which matched the result I got most
            frequently on the SlugOS slug itself, but one of the copies gave me a
            different result.

            So it appears to me that the actual file is being misread in some way.
            Could the ext3 driver be causing the problems? The variable md5
            results makes it sound like there's a read problem on the slug, before
            the file even goes out onto the network. The incorrect md5 on a file
            copied ONTO the SlugOS slug makes it sound like the error might also
            affect writes. I'm tempted to try a different filesystem type - any
            recommendation? No way of eliminating the USB driver though.

            Thomas, could you find a couple of GB of 100+MB files on your SlugOS
            slug and run a couple of md5sum passes over them, pipe the ouputs to
            files & then diff them to look for changes? See if you're seeing the
            same thing as me.

            Thanks everyone

            Jon
          Your message has been successfully submitted and would be delivered to recipients shortly.