Loading ...
Sorry, an error occurred while loading the content.

Re: Life after NSLU2?

Expand Messages
  • docbillnet
    ... page is ... won t be ... The man page says the checksums are always computed to compare the transfered files. Since you used -n, there was no transfer, so
    Message 1 of 18 , Apr 1, 2008
    • 0 Attachment
      --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@...> wrote:
      >
      > On Mon, Mar 31, 2008 at 10:09 AM, docbillnet <yahoo@...> wrote:
      >
      > > --- In nslu2-general@yahoogroups.com, "bloedmann999"
      > > <Brian_Dorling@> wrote:
      > 5. If I understand the process correctly, it appears that the MAN
      page is
      > wrong in regards to checksum always being calculated for rsync. It
      won't be
      > the first time a man page is wrong or clear as mud. :-)

      The man page says the checksums are always computed to compare the
      transfered files. Since you used -n, there was no transfer, so there
      was no checksum. If it doesn't write the temp file, it has nothing to
      compute the checksum on.

      Bill
    • Rob Lockhart
      ... I just tried it again, and with touch -t 200804010026.00 100MiB_file on both sides, with *different CRCs*, there was no transfer, even without the -n
      Message 2 of 18 , Apr 1, 2008
      • 0 Attachment
        On Tue, Apr 1, 2008 at 1:42 PM, docbillnet <yahoo@...> wrote:

        > --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@...> wrote:
        > >
        > > On Mon, Mar 31, 2008 at 10:09 AM, docbillnet <yahoo@...> wrote:
        > >
        > > > --- In nslu2-general@yahoogroups.com, "bloedmann999"
        > > > <Brian_Dorling@> wrote:
        > > 5. If I understand the process correctly, it appears that the MAN
        > page is
        > > wrong in regards to checksum always being calculated for rsync.
        >
        > The man page says the checksums are always computed to compare the
        > transfered files. Since you used -n, there was no transfer, so there
        > was no checksum. If it doesn't write the temp file, it has nothing to
        > compute the checksum on.
        >

        I just tried it again, and with "touch -t 200804010026.00 100MiB_file" on
        both sides, with *different CRCs*, there was no transfer, even without the
        "-n" switch. No temp file created, no checksum generated, as there was no
        transfer.

        rob@NSLU2:~$ md5sum -b 100MiB_file
        62b2c27a678fd3bf07418c9f4f161c93 *100MiB_file
        rob@NSLU2:~$ touch -t 200804010026.00 100MiB_file

        [rob@Linux ~]$ md5sum -b 100MiB_file
        2ed67109979cec75d1b155581cdb7cf4 *100MiB_file
        [rob@Linux ~]$ touch -t 200804010026.00 100MiB_file

        [rob@Linux ~]$ date; rsync -va --delete --progress --stats
        rob@NSLU2:100MiB_file
        100MiB_file ; date
        Tue Apr 1 22:05:01 EDT 2008
        receiving file list ...
        1 file to consider

        Number of files: 1
        Number of files transferred: 0
        Total file size: 104857600 bytes
        Total transferred file size: 0 bytes
        Literal data: 0 bytes
        Matched data: 0 bytes
        File list size: 62
        File list generation time: 0.012 seconds
        File list transfer time: 0.000 seconds
        Total bytes sent: 20
        Total bytes received: 82

        sent 20 bytes received 82 bytes 40.80 bytes/sec
        total size is 104857600 speedup is 1028015.69
        Tue Apr 1 22:05:03 EDT 2008

        [rob@Linux ~]$ md5sum -b 100MiB_file
        2ed67109979cec75d1b155581cdb7cf4 *100MiB_file

        rob@NSLU2:~$ md5sum -b 100MiB_file
        62b2c27a678fd3bf07418c9f4f161c93 *100MiB_file

        Assuming the checksum is generated while the file is being transferred, that
        doesn't seem to be the determining factor for whether to transfer files of
        identical UID/date/time/size (but different content). It appears that
        without the "-c" switch, the CRC is not calculated on files for which there
        is no pending transfer. I'm not sure what would be gained in calculating
        the CRC on a file during transfer, perhaps only to ensure that it matches
        the destination file CRC when being created? Indeed, the man page is
        correct regarding CRC of transferred files (based on criterion given to
        rsync with which to compare the files).

        -Rob

        P.S. Apologies for the diversion (thread hijacking), folks. Should've
        changed the subject to rsync or similar.


        [Non-text portions of this message have been removed]
      • docbillnet
        ... 100MiB_file on ... without the ... was no ... As predicted. Exactly what I said, the checksum is always computed for transfered files. I am glad you
        Message 3 of 18 , Apr 2, 2008
        • 0 Attachment
          --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@...> wrote:
          >
          > On Tue, Apr 1, 2008 at 1:42 PM, docbillnet <yahoo@...> wrote:
          >
          > > --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@>
          wrote:
          > > The man page says the checksums are always computed to compare the
          > > transfered files. Since you used -n, there was no transfer, so there
          > > was no checksum. If it doesn't write the temp file, it has nothing to
          > > compute the checksum on.
          > >
          >
          > I just tried it again, and with "touch -t 200804010026.00
          100MiB_file" on
          > both sides, with *different CRCs*, there was no transfer, even
          without the
          > "-n" switch. No temp file created, no checksum generated, as there
          was no
          > transfer.

          As predicted. Exactly what I said, the checksum is always computed
          for transfered files. I am glad you could validate that the manual
          page is correct.

          > is no pending transfer. I'm not sure what would be gained in
          calculating
          > the CRC on a file during transfer, perhaps only to ensure that it
          matches
          > the destination file CRC when being created? Indeed, the man page is

          I am assuming that the same code is used to compute checksums while
          reading regardless if the file transfer is local or remote. The same
          code is also probably used to compute checksums when saving,
          regardless of transfer mode. I can see absolutely no advantage in
          computing a checksum as your read a file and a checksum as you write a
          file locally. If the checksum is changing in your memory blocks, then
          no amount of sanity checking will help... I can see the advantage
          across a socket or a pipe. TCP-IP packets only use a very small
          checksum (I think it is one byte). Which means if you have enough
          errors occur you are almost guaranteed that one of them will be
          received as valid data. When you consider people will use rsync to
          copy many terabytes of data, that means they would regularly corrupt
          files with even the rarest of network errors if not for the additional
          checksums.

          Bill
        Your message has been successfully submitted and would be delivered to recipients shortly.