Loading ...
Sorry, an error occurred while loading the content.

54Unison beta-release updated

Expand Messages
  • Benjamin Pierce
    Jun 27, 2010
    • 0 Attachment
      This beta-release improves the previous beta-release (version 2.40.1) with several bugfixes and speed improvements.

      Enjoy,

      - Benjamin (on behalf of the Unison team)


      Download address:
      http://www.cis.upenn.edu/~bcpierce/unison/download.html

      Changes in Version 2.40.16

      Changes since 2.40.1:
      * Added "BelowPath" patterns, that match a path as well as all paths
      below (convenient to use with nodeletion,update,creationpartial
      preferences)
      * Added a "fat" preference that makes Unison use the right options
      when one of the replica is on a FAT filesystem.
      * Allow "prefer/force=newer" even when not synchronizing
      modification times. (The reconciler will not be aware of the
      modification time of unchanged files, so the synchronization
      choices of Unison can be different from when "times=true", but the
      behavior remains sane: changed files with the most recent
      modification time will be propagated.)
      * Minor fixes and improvements:
      + Compare filenames up to decomposition in case sensitive mode
      when one host is running MacOSX and the unicode preference is
      set to true.
      + Rsync: somewhat faster compressor
      + Make Unicode the default on all architectures (it was only
      the default when a Mac OS X or Windows machine was involved).

      Changes since 2.32:
      * Major enhancement: Unicode support.
      + Unison should now handle unicode filenames correctly on all
      platforms.
      + This functionality is controlled by a new preference unicode.
      + Unicode mode is now the default when one of the hosts is
      under Windows or MacOS. This may make upgrades a bit more
      painful (the archives cannot be reused), but this is a much
      saner default.
      * Partial transfer of directories. If an error occurs while
      transferring a directory, the part transferred so far is copied
      into place (and the archives are updated accordingly). The
      "maxerrors" preference controls how many transfer error Unison
      will accept before stopping the transfer of a directory (by
      default, only one). This makes it possible to transfer most of a
      directory even if there are some errors. Currently, only the first
      error is reported by the GUIs.
      Also, allow partial transfer of a directory when there was an
      error deep inside this directory during update detection. At the
      moment, this is only activated with the text and GTK UIs, which
      have been modified so that they show that the transfer is going to
      be partial and so that they can display all errors.
      * Improvement to the code for resuming directory transfers:
      + if a file was not correctly transferred (or the source has
      been modified since, with unchanged size), Unison performs a
      new transfer rather than failing
      + spurious files are deleted (this can happen if a file is
      deleted on the source replica before resuming the transfer;
      not deleting the file would result in it reappearing on the
      target replica)
      * Experimental streaming protocol for transferring file contents
      (can be disabled by setting the directive "stream" to false): file
      contents is transfered asynchronously (without waiting for a
      response from the destination after each chunk sent) rather than
      using the synchronous RPC mechanism. As a consequence:
      + Unison now transfers the contents of a single file at a time
      (Unison used to transfer several contents simultaneously in
      order to hide the connection latency.)
      + the transfer of large files uses the full available bandwidth
      and is not slowed done due to the connection latency anymore
      + we get performance improvement for small files as well by
      scheduling many files simultaneously (as scheduling a file
      for transfer consume little ressource: it does not mean
      allocating a large buffer anymore)
      * Changes to the internal implementation of the rsync algorithm:
      + use longer blocks for large files (the size of a block is the
      square root of the size of the file for large files);
      + transmit less checksum information per block (we still have
      less than one chance in a hundred million of transferring a
      file incorrectly, and Unison will catch any transfer error
      when fingerprinting the whole file)
      + avoid transfer overhead (which was 4 bytes per block)
      For a 1G file, the first optimization saves a factor 50 on the
      amount of data transferred from the target to the source (blocks
      are 32768 bytes rather than just 700 bytes). The two other
      optimizations save another factor of 2 (from 24 bytes per block
      down to 10).
      * Implemented an on-disk file fingerprint cache to speed-up update
      detection after a crash: this way, Unison does not have do
      recompute all the file fingerprints from scratch.
      + When Unison detects that the archive case-sensitivity mode
      does not match the current settings, it populates the
      fingerprint cache using the archive contents. This way,
      changing the case-sensitivity mode should be reasonably fast.
      * New preferences "noupdate=root", "nodeletion=root",
      "nocreation=root" that prevent Unison from performing files
      updates, deletions or creations on the given root. Also 'partial'
      versions of 'noupdate', 'nodeletion' and 'nocreation'
      * Limit the number of simultaneous external copy program ("copymax"
      preference)
      * New "links" preference. When set to false, Unison will report an
      error on symlinks during update detection. (This is the default
      when one host is running Windows but not Cygwin.) This is better
      than failing during propagation.
      * Added a preference "halfduplex" to force half-duplex communication
      with the server. This may be useful on unreliable links (as a more
      efficient alternative to "maxthreads = 1").
      * Renamed preference "pretendwin" to "ignoreinodenumbers" (an alias
      is kept for backwards compatibility).
      * Ignore one-second differences when synchronizing modification
      time. (Technically, this is an incompatible archive format change,
      but it is backward compatible. To trigger a problem, a user would
      have to synchronize modification times on a filesystem with a
      two-second granularity and then downgrade to a previous version of
      Unison, which does not work well in such a case. Thus, it does not
      seem worthwhile to increment the archive format number, which
      would impact all users.)
      * Do not keep many files simultaneously opened anymore when the
      rsync algorithm is in use.
      * Add "ignorearchives" preference to ignore existing archives (to
      avoid forcing users to delete them manually, in situations where
      one archive has gotten deleted or corrupted).
      * Mac OS
      + fixed rsync bug which could result in an "index out of
      bounds" error when transferring resource forks.
      + Fixed bug which made Unison ignore finder information and
      resource fork when compiled to 64bit on Mac OSX.
      + should now be 64 bit clean (the Growl framework is not up to
      date, though)
      + Made the bridge between Objective C and Ocaml code GC
      friendly (it was allocating ML values and putting them in an
      array which was not registered with the GC)
      + use darker grey arrows (patch contributed by Eric Y. Kow)
      * GTK user interface
      + assistant for creating profiles
      + profile editor
      + pop up a summary window when the replicas are not fully
      synchronized after transport
      + display estimated remaining time and transfer rate on the
      progress bar
      + allow simultaneous selection of several items
      + Do not reload the preference file before a new update
      detection if it is unchanged
      + disabled scrolling to the first unfinished item during
      transport. It goes way too fast when lot of small files are
      synchronized, and it makes it impossible to browse the file
      list during transport.
      + take into account the "height" preference again
      + the internal list of selected reconciler item was not always
      in sync with what was displayed (GTK bug?); workaround
      implemented
      + Do not display "Looking for change" messages during
      propagation (when checking the targe is unchanged) but only
      during update detection
      + Apply patch to fix some crashes in the OSX GUI, thanks to
      Onne Gorter.
      * Text UI
      + During update detection, display status by updating a single
      line rather than generating a new line of output every so
      often. Should be less confusing.
      * Windows
      + Fastcheck is now the default under Windows. People mostly use
      NTFS nowadays and the Unicode API provides an equivalent to
      inode numbers for this filesystem.
      + Only use long UNC path for accessing replicas (as '..' is not
      handled with this format of paths, but can be useful)
      + Windows text UI: now put the console into UTF-8 output mode.
      This is the right thing to do when in Unicode mode, and is no
      worse than what we had previously otherwise (the console use
      some esoteric encoding by default). This only works when
      using a Unicode font instead of the default raster font.
      + Don't get the home directory from environment variable HOME
      under Windows (except for Cygwin binaries): we don't want the
      behavior of Unison to depends on whether it is run from a
      Cygwin shell (where HOME is set) or in any other way (where
      HOME is usually not set).
      * Miscellaneous fixes and improvements
      + Made a server waiting on a socket more resilient to
      unexpected lost connections from the client.
      + Small patch to property setting code suggested by Ulrich
      Gernkow.
      + Several fixes to the change transfer functions (both the
      internal ones and external transfers using rsync). In
      particular, limit the number of simultaneous transfer using
      an rsync (as the rsync algorithm can use a large amount of
      memory when processing huge files)
      + Keep track of which file contents are being transferred, and
      delay the transfer of a file when another file with the same
      contents is currently being transferred. This way, the second
      transfer can be skipped and replaced by a local copy.
      + Experimental update detection optimization: do not read the
      contents of unchanged directories
      + When a file transfer fails, turn off fastcheck for this file
      on the next sync.
      + Fixed bug with case insensitive mode on a case sensitive
      filesystem:
      o if file "a/a" is created on one replica and directory
      "A" is created on the other, the file failed to be
      synchronized the first time Unison is run afterwards, as
      Unison uses the wrong path "a/a" (if Unison is run
      again, the directories are in the archive, so the right
      path is used);
      o if file "a" appears on one replica and file "A" appears
      on the other with different contents, Unison was unable
      to synchronize them.
      + Improved error reporting when the destination is updated
      during synchronization: Unison now tells which file has been
      updated, and how.
      + Limit the length of temporary file names
      + Case sensitivity information put in the archive (in a
      backward compatible way) and checked when the archive is
      loaded
      + Got rid of the 16mb marshalling limit by marshalling to a
      bigarray.
      + Resume copy of partially transferred files.

      Changes since 2.31:
      * Small user interface changes
      + Small change to text UI "scanning..." messages, to print just
      directories (hopefully making it clearer that individual
      files are not necessarily being fingerprinted).
      * Minor fixes and improvements:
      + Ignore one hour differences when deciding whether a file may
      have been updated. This avoids slow update detection after
      daylight saving time changes under Windows. This makes Unison
      slightly more likely to miss an update, but it should be safe
      enough.
      + Fix a small bug that was affecting mainly windows users. We
      need to commit the archives at the end of the sync even if
      there are no updates to propagate because some files (in
      fact, if we've just switched to DST on windows, a LOT of
      files) might have new modtimes in the archive. (Changed the
      text UI only. It's less clear where to change the GUI.)
      + Don't delete the temp file when a transfer fails due to a
      fingerprint mismatch (so that we can have a look and see
      why!) We've also added more debugging code togive more
      informative error messages when we encounter the dreaded and
      longstanding "assert failed during file transfer" bug
      + Incorrect paths ("path" directive) now result in an error
      update item rather than a fatal error.
      + Create parent directories (with correct permissions) during
      transport for paths which point to non-existent locations in
      the destination replica.

      Changes since 2.27:
      * If Unison is interrupted during a directory transfer, it will now
      leave the partially transferred directory intact in a temporary
      location. (This maintains the invariant that new files/directories
      are transferred either completely or not at all.) The next time
      Unison is run, it will continue filling in this temporary
      directory, skipping transferring files that it finds are already
      there.
      * We've added experimental support for invoking an external file
      transfer tool for whole-file copies instead of Unison's built-in
      transfer protocol. Three new preferences have been added:
      + copyprog is a string giving the name (and command-line
      switches, if needed) of an external program that can be used
      to copy large files efficiently. By default, rsync is
      invoked, but other tools such as scp can be used instead by
      changing the value of this preference. (Although this is not
      its primary purpose, rsync is actually a pretty fast way of
      copying files that don't already exist on the receiving
      host.) For files that do already exist on (but that have been
      changed in one replica), Unison will always use its built-in
      implementation of the rsync algorithm.
      + Added a "copyprogrest" preference, so that we can give
      different command lines for invoking the external copy
      utility depending on whether a partially transferred file
      already exists or not. (Rsync doesn't seem to care about
      this, but other utilities may.)
      + copythreshold is an integer (-1 by default), indicating above
      what filesize (in megabytes) Unison should use the external
      copying utility specified by copyprog. Specifying 0 will
      cause ALL copies to use the external program; a negative
      number will prevent any files from using it. (Default is -1.)
      Thanks to Alan Schmitt for a huge amount of hacking and to an
      anonymous sponsor for suggesting and underwriting this extension.
      * Small improvements:
      + Added a new preference, dontchmod. By default, Unison uses
      the chmod system call to set the permission bits of files
      after it has copied them. But in some circumstances (and
      under some operating systems), the chmod call always fails.
      Setting this preference completely prevents Unison from ever
      calling chmod.
      + Don't ignore files that look like backup files if the
      backuplocation preference is set to central
      + Shortened the names of several preferences. The old names are
      also still supported, for backwards compatibility, but they
      do not appear in the documentation.
      + Lots of little documentation tidying. (In particular,
      preferences are separated into Basic and Advanced! This
      should hopefully make Unison a little more approachable for
      new users.
      + Unison can sometimes fail to transfer a file, giving the
      unhelpful message "Destination updated during
      synchronization" even though the file has not been changed.
      This can be caused by programs that change either the file's
      contents or the file's extended attributes without changing
      its modification time. It's not clear what is the best fix
      for this - it is not Unison's fault, but it makes Unison's
      behavior puzzling - but at least Unison can be more helpful
      about suggesting a workaround (running once with fastcheck
      set to false). The failure message has been changed to give
      this advice.
      + Further improvements to the OS X GUI (thanks to Alan Schmitt
      and Craig Federighi).
      * Very preliminary support for triggering Unison from an external
      filesystem-watching utility. The current implementation is very
      simple, not efficient, and almost completely untested--not ready
      for real users. But if someone wants to help improve it (e.g., by
      writing a filesystem watcher for your favorite OS), please make
      yourself known!
      On the Unison side, the new behavior is very simple:
      + use the text UI
      + start Unison with the command-line flag "-repeat FOO", where
      FOO is name of a file where Unison should look for
      notifications of changes
      + when it starts up, Unison will read the whole contents of
      this file (on both hosts), which should be a
      newline-separated list of paths (relative to the root of the
      synchronization) and synchronize just these paths, as if it
      had been started with the "-path=xxx" option for each one of
      them
      + when it finishes, it will sleep for a few seconds and then
      examine the watchfile again; if anything has been added, it
      will read the new paths, synchronize them, and go back to
      sleep
      + that's it!
      To use this to drive Unison "incrementally," just start it in this
      mode and start up a tool (on each host) to watch for new changes
      to the filesystem and append the appropriate paths to the
      watchfile. Hopefully such tools should not be too hard to write.
      * Bug fixes:
      + Fixed a bug that was causing new files to be created with
      permissions 0x600 instead of using a reasonable default (like
      0x644), if the 'perms' flag was set to 0. (Bug reported by
      Ben Crowell.)
      + Follow maxthreads preference when transferring directories.

      Changes since 2.17:
      * Major rewrite and cleanup of the whole Mac OS X graphical user
      interface by Craig Federighi. Thanks, Craig!!!
      * Small fix to ctime (non-)handling in update detection under
      windows with fastcheck.
      * Several small fixes to the GTK2 UI to make it work better under
      Windows [thanks to Karl M for these].
      * The backup functionality has been completely rewritten. The
      external interface has not changed, but numerous bugs, irregular
      behaviors, and cross-platform inconsistencies have been corrected.
      * The Unison project now accepts donations via PayPal. If you'd like
      to donate, you can find a link to the donation page on the Unison
      home page (http://www.cis.upenn.edu/ bcpierce/unison/lists.html).
      * Some important safety improvements:
      + Added a new mountpoint preference, which can be used to
      specify a path that must exist in both replicas at the end of
      update detection (otherwise Unison aborts). This can be used
      to avoid potentially dangerous situations when Unison is used
      with removable media such as external hard drives and compact
      flash cards.
      + The confirmation of "big deletes" is now controlled by a
      boolean preference confirmbigdeletes. Default is true, which
      gives the same behavior as previously. (This functionality is
      at least partly superceded by the mountpoint preference, but
      it has been left in place in case it is useful to some
      people.)
      + If Unison is asked to "follow" a symbolic link but there is
      nothing at the other end of the link, it will now flag this
      path as an error, rather than treating the symlink itself as
      missing or deleted. This avoids a potentially dangerous
      situation where a followed symlink points to an external
      filesystem that might be offline when Unison is run
      (whereupon Unison would cheerfully delete the corresponding
      files in the other replica!).
      * Smaller changes:
      + Added forcepartial and preferpartial preferences, which
      behave like force and prefer but can be specified on a
      per-path basis. [Thanks to Alan Schmitt for this.]
      + A bare-bones self test feature was added, which runs unison
      through some of its paces and checks that the results are as
      expected. The coverage of the tests is still very limited,
      but the facility has already been very useful in debugging
      the new backup functionality (especially in exposing some
      subtle cross-platform issues).
      + Refined debugging code so that the verbosity of individual
      modules can be controlled separately. Instead of just putting
      '-debug verbose' on the command line, you can put '-debug
      update+', which causes all the extra messages in the Update
      module, but not other modules, to be printed. Putting '-debug
      verbose' causes all modules to print with maximum verbosity.
      + Removed mergebatch preference. (It never seemed very useful,
      and its semantics were confusing.)
      + Rewrote some of the merging functionality, for better
      cooperation with external Harmony instances.
      + Changed the temp file prefix from .# to .unison.
      + Compressed the output from the text user interface
      (particularly when run with the -terse flag) to make it
      easier to interpret the results when Unison is run several
      times in succession from a script.
      + Diff and merge functions now work under Windows.
      + Changed the order of arguments to the default diff command
      (so that the + and - annotations in diff's output are
      reversed).
      + Added .mpp files to the "never fastcheck" list (like .xls
      files).
      * Many small bugfixes, including:
      + Fixed a longstanding bug regarding fastcheck and daylight
      saving time under Windows when Unison is set up to
      synchronize modification times. (Modification times cannot be
      updated in the archive in this case, so we have to ignore one
      hour differences.)
      + Fixed a bug that would occasionally cause the archives to be
      left in non-identical states on the two hosts after
      synchronization.
      + Fixed a bug that prevented Unison from communicating
      correctly between 32- and 64-bit architectures.
      + On windows, file creation times are no longer used as a proxy
      for inode numbers. (This is unfortunate, as it makes
      fastcheck a little less safe. But it turns out that file
      creation times are not reliable under Windows: if a file is
      removed and a new file is created in its place, the new one
      will sometimes be given the same creation date as the old
      one!)
      + Set read-only file to R/W on OSX before attempting to change
      other attributes.
      + Fixed bug resulting in spurious "Aborted" errors during
      transport (thanks to Jerome Vouillon)
      + Enable diff if file contents have changed in one replica, but
      only properties in the other.
      + Removed misleading documentation for 'repeat' preference.
      + Fixed a bug in merging code where Unison could sometimes
      deadlock with the external merge program, if the latter
      produced large amounts of output.
      + Workaround for a bug compiling gtk2 user interface against
      current versions of gtk2+ libraries.
      + Added a better error message for "ambiguous paths".
      + Squashed a longstanding bug that would cause file transfer to
      fail with the message "Failed: Error in readWrite: Is a
      directory."
      + Replaced symlinks with copies of their targets in the Growl
      framework in src/uimac. This should make the sources easier
      to check out from the svn repository on WinXP systems.
      + Added a workaround (suggested by Karl M.) for the problem
      discussed on the unison users mailing list where, on the
      Windows platform, the server would hang when transferring
      files. I conjecture that the problem has to do with the RPC
      mechanism, which was used to make a call back from the server
      to the client (inside the Trace.log function) so that the log
      message would be appended to the log file on the client. The
      workaround is to dump these messages (about when
      xferbycopying shortcuts are applied and whether they succeed)
      just to the standard output of the Unison process, not to the
      log file.

      Changes since 2.13.0:
      * The features for performing backups and for invoking external
      merge programs have been completely rewritten by Stephane Lescuyer
      (thanks, Stephane!). The user-visible functionality should not
      change, but the internals have been rationalized and there are a
      number of new features. See the manual (in particular, the
      description of the backupXXX preferences) for details.
      * Incorporated patches for ipv6 support, contributed by Samuel
      Thibault. (Note that, due to a bug in the released OCaml 3.08.3
      compiler, this code will not actually work with ipv6 unless
      compiled with the CVS version of the OCaml compiler, where the bug
      has been fixed; however, ipv4 should continue to work normally.)
      * OSX interface:
      + Incorporated Ben Willmore's cool new icon for the Mac UI.
      * Small fixes:
      + Fixed off by one error in month numbers (in printed dates)
      reported by Bob Burger

      Changes since 2.12.0:
      * New convention for release numbering: Releases will continue to be
      given numbers of the form X.Y.Z, but, from now on, just the major
      version number (X.Y) will be considered significant when checking
      compatibility between client and server versions. The third
      component of the version number will be used only to identify
      "patch levels" of releases.
      This change goes hand in hand with a change to the procedure for
      making new releases. Candidate releases will initially be given
      "beta release" status when they are announced for public
      consumption. Any bugs that are discovered will be fixed in a
      separate branch of the source repository (without changing the
      major version number) and new tarballs re-released as needed. When
      this process converges, the patched beta version will be dubbed
      stable.
      * Warning (failure in batch mode) when one path is completely
      emptied. This prevents Unison from deleting everything on one
      replica when the other disappear.
      * Fix diff bug (where no difference is shown the first time the diff
      command is given).
      * User interface changes:
      + Improved workaround for button focus problem (GTK2 UI)
      + Put leading zeroes in date fields
      + More robust handling of character encodings in GTK2 UI
      + Changed format of modification time displays, from modified
      at hh:mm:ss on dd MMM, yyyy to modified on yyyy-mm-dd
      hh:mm:ss
      + Changed time display to include seconds (so that people on
      FAT filesystems will not be confused when Unison tries to
      update a file time to an odd number of seconds and the
      filesystem truncates it to an even number!)
      + Use the diff "-u" option by default when showing differences
      between files (the output is more readable)
      + In text mode, pipe the diff output to a pager if the
      environment variable PAGER is set
      + Bug fixes and cleanups in ssh password prompting. Now works
      with the GTK2 UI under Linux. (Hopefully the Mac OS X one is
      not broken!)
      + Include profile name in the GTK2 window name
      + Added bindings ',' (same as '<') and '.' (same as '>') in the
      GTK2 UI
      * Mac GUI:
      + actions like < and > scroll to the next item as necessary.
      + Restart has a menu item and keyboard shortcut (command-R).
      + Added a command-line tool for Mac OS X. It can be installed
      from the Unison menu.
      + New icon.
      + Handle the "help" command-line argument properly.
      + Handle profiles given on the command line properly.
      + When a profile has been selected, the profile dialog is
      replaced by a "connecting" message while the connection is
      being made. This gives better feedback.
      + Size of left and right columns is now large enough so that
      "PropsChanged" is not cut off.
      * Minor changes:
      + Disable multi-threading when both roots are local
      + Improved error handling code. In particular, make sure all
      files are closed in case of a transient failure
      + Under Windows, use $UNISON for home directory as a last
      resort (it was wrongly moved before $HOME and $USERPROFILE in
      Unison 2.12.0)
      + Reopen the logfile if its name changes (profile change)
      + Double-check that permissions and modification times have
      been properly set: there are some combination of OS and
      filesystem on which setting them can fail in a silent way.
      + Check for bad Windows filenames for pure Windows
      synchronization also (not just cross architecture
      synchronization). This way, filenames containing backslashes,
      which are not correctly handled by unison, are rejected right
      away.
      + Attempt to resolve issues with synchronizing modification
      times of read-only files under Windows
      + Ignore chmod failures when deleting files
      + Ignore trailing dots in filenames in case insensitive mode
      + Proper quoting of paths, files and extensions ignored using
      the UI
      + The strings CURRENT1 and CURRENT2 are now correctly
      substitued when they occur in the diff preference
      + Improvements to syncing resource forks between Macs via a
      non-Mac system.