Loading ...
Sorry, an error occurred while loading the content.

Re: [nslu2-general] Re: Life after NSLU2?

Expand Messages
  • Rob Lockhart
    ... I just tried it again, and with touch -t 200804010026.00 100MiB_file on both sides, with *different CRCs*, there was no transfer, even without the -n
    Message 1 of 18 , Apr 1 7:34 PM
    View Source
    • 0 Attachment
      On Tue, Apr 1, 2008 at 1:42 PM, docbillnet <yahoo@...> wrote:

      > --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@...> wrote:
      > >
      > > On Mon, Mar 31, 2008 at 10:09 AM, docbillnet <yahoo@...> wrote:
      > >
      > > > --- In nslu2-general@yahoogroups.com, "bloedmann999"
      > > > <Brian_Dorling@> wrote:
      > > 5. If I understand the process correctly, it appears that the MAN
      > page is
      > > wrong in regards to checksum always being calculated for rsync.
      >
      > The man page says the checksums are always computed to compare the
      > transfered files. Since you used -n, there was no transfer, so there
      > was no checksum. If it doesn't write the temp file, it has nothing to
      > compute the checksum on.
      >

      I just tried it again, and with "touch -t 200804010026.00 100MiB_file" on
      both sides, with *different CRCs*, there was no transfer, even without the
      "-n" switch. No temp file created, no checksum generated, as there was no
      transfer.

      rob@NSLU2:~$ md5sum -b 100MiB_file
      62b2c27a678fd3bf07418c9f4f161c93 *100MiB_file
      rob@NSLU2:~$ touch -t 200804010026.00 100MiB_file

      [rob@Linux ~]$ md5sum -b 100MiB_file
      2ed67109979cec75d1b155581cdb7cf4 *100MiB_file
      [rob@Linux ~]$ touch -t 200804010026.00 100MiB_file

      [rob@Linux ~]$ date; rsync -va --delete --progress --stats
      rob@NSLU2:100MiB_file
      100MiB_file ; date
      Tue Apr 1 22:05:01 EDT 2008
      receiving file list ...
      1 file to consider

      Number of files: 1
      Number of files transferred: 0
      Total file size: 104857600 bytes
      Total transferred file size: 0 bytes
      Literal data: 0 bytes
      Matched data: 0 bytes
      File list size: 62
      File list generation time: 0.012 seconds
      File list transfer time: 0.000 seconds
      Total bytes sent: 20
      Total bytes received: 82

      sent 20 bytes received 82 bytes 40.80 bytes/sec
      total size is 104857600 speedup is 1028015.69
      Tue Apr 1 22:05:03 EDT 2008

      [rob@Linux ~]$ md5sum -b 100MiB_file
      2ed67109979cec75d1b155581cdb7cf4 *100MiB_file

      rob@NSLU2:~$ md5sum -b 100MiB_file
      62b2c27a678fd3bf07418c9f4f161c93 *100MiB_file

      Assuming the checksum is generated while the file is being transferred, that
      doesn't seem to be the determining factor for whether to transfer files of
      identical UID/date/time/size (but different content). It appears that
      without the "-c" switch, the CRC is not calculated on files for which there
      is no pending transfer. I'm not sure what would be gained in calculating
      the CRC on a file during transfer, perhaps only to ensure that it matches
      the destination file CRC when being created? Indeed, the man page is
      correct regarding CRC of transferred files (based on criterion given to
      rsync with which to compare the files).

      -Rob

      P.S. Apologies for the diversion (thread hijacking), folks. Should've
      changed the subject to rsync or similar.


      [Non-text portions of this message have been removed]
    • docbillnet
      ... 100MiB_file on ... without the ... was no ... As predicted. Exactly what I said, the checksum is always computed for transfered files. I am glad you
      Message 2 of 18 , Apr 2 6:32 AM
      View Source
      • 0 Attachment
        --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@...> wrote:
        >
        > On Tue, Apr 1, 2008 at 1:42 PM, docbillnet <yahoo@...> wrote:
        >
        > > --- In nslu2-general@yahoogroups.com, "Rob Lockhart" <rlockhar@>
        wrote:
        > > The man page says the checksums are always computed to compare the
        > > transfered files. Since you used -n, there was no transfer, so there
        > > was no checksum. If it doesn't write the temp file, it has nothing to
        > > compute the checksum on.
        > >
        >
        > I just tried it again, and with "touch -t 200804010026.00
        100MiB_file" on
        > both sides, with *different CRCs*, there was no transfer, even
        without the
        > "-n" switch. No temp file created, no checksum generated, as there
        was no
        > transfer.

        As predicted. Exactly what I said, the checksum is always computed
        for transfered files. I am glad you could validate that the manual
        page is correct.

        > is no pending transfer. I'm not sure what would be gained in
        calculating
        > the CRC on a file during transfer, perhaps only to ensure that it
        matches
        > the destination file CRC when being created? Indeed, the man page is

        I am assuming that the same code is used to compute checksums while
        reading regardless if the file transfer is local or remote. The same
        code is also probably used to compute checksums when saving,
        regardless of transfer mode. I can see absolutely no advantage in
        computing a checksum as your read a file and a checksum as you write a
        file locally. If the checksum is changing in your memory blocks, then
        no amount of sanity checking will help... I can see the advantage
        across a socket or a pipe. TCP-IP packets only use a very small
        checksum (I think it is one byte). Which means if you have enough
        errors occur you are almost guaranteed that one of them will be
        received as valid data. When you consider people will use rsync to
        copy many terabytes of data, that means they would regularly corrupt
        files with even the rarest of network errors if not for the additional
        checksums.

        Bill
      Your message has been successfully submitted and would be delivered to recipients shortly.