New Unison beta-release (2.40)

On behalf of the Unison team, I'm delight to announce the availability of a=
 new beta-release, version 2.40:

   http://www.cis.upenn.edu/~bcpierce/unison/download.html

Source code is available now for those that want to compile it themselves. =
 Binaries for various architectures will appear in due course.

Major kudos to J=E9rome Vouillon for the lion's share of the hacking on the=
 new features!

Enjoy,

    Benjamin


Changes since 2.32:
  * Major enhancement: Unicode support.
       + Unison should now handle unicode filenames correctly on all
         platforms.
       + This functionality is controlled by a new preference unicode.
       + Unicode mode is now the default on all architectures.=20
         This may make upgrades a bit more painful (the archives=20
         cannot be reused), but it is the right default.
  * Partial transfer of directories. If an error occurs while
    transferring a directory, the part transferred so far is copied
    into place (and the archives are updated accordingly). The
    "maxerrors" preference controls how many transfer error Unison
    will accept before stopping the transfer of a directory (by
    default, only one). This makes it possible to transfer most of a
    directory even if there are some errors. Currently, only the first
    error is reported by the GUIs.
    Also, allow partial transfer of a directory when there was an
    error deep inside this directory during update detection. At the
    moment, this is only activated with the text and GTK UIs, which
    have been modified so that they show that the transfer is going to
    be partial and so that they can display all errors.
  * Improvement to the code for resuming directory transfers:
       + if a file was not correctly transferred (or the source has
         been modified since, with unchanged size), Unison performs a
         new transfer rather than failing
       + spurious files are deleted (this can happen if a file is
         deleted on the source replica before resuming the transfer;
         not deleting the file would result in it reappearing on the
         target replica)
  * Experimental streaming protocol for transferring file contents
    (can be disabled by setting the directive "stream" to false): file
    contents is transfered asynchronously (without waiting for a
    response from the destination after each chunk sent) rather than
    using the synchronous RPC mechanism. As a consequence:
       + Unison now transfers the contents of a single file at a time
         (Unison used to transfer several contents simultaneously in
         order to hide the connection latency.)
       + the transfer of large files uses the full available bandwidth
         and is not slowed done due to the connection latency anymore
       + we get performance improvement for small files as well by
         scheduling many files simultaneously (as scheduling a file
         for transfer consume little ressource: it does not mean
         allocating a large buffer anymore)
  * Changes to the internal implementation of the rsync algorithm:
       + use longer blocks for large files (the size of a block is the
         square root of the size of the file for large files);
       + transmit less checksum information per block (we still have
         less than one chance in a hundred million of transferring a
         file incorrectly, and Unison will catch any transfer error
         when fingerprinting the whole file)
       + avoid transfer overhead (which was 4 bytes per block)
    For a 1G file, the first optimization saves a factor 50 on the
    amount of data transferred from the target to the source (blocks
    are 32768 bytes rather than just 700 bytes). The two other
    optimizations save another factor of 2 (from 24 bytes per block
    down to 10).
  * Implemented an on-disk file fingerprint cache to speed-up update
    detection after a crash: this way, Unison does not have do
    recompute all the file fingerprints from scratch.
       + When Unison detects that the archive case-sensitivity mode
         does not match the current settings, it populates the
         fingerprint cache using the archive contents. This way,
         changing the case-sensitivity mode should be reasonably fast.
  * New preferences "noupdate=3Droot", "nodeletion=3Droot",
    "nocreation=3Droot" that prevent Unison from performing files
    updates, deletions or creations on the given root. Also 'partial'
    versions of 'noupdate', 'nodeletion' and 'nocreation'
  * Limit the number of simultaneous external copy program ("copymax"
    preference)
  * New "links" preference. When set to false, Unison will report an
    error on symlinks during update detection. (This is the default
    when one host is running Windows but not Cygwin.) This is better
    than failing during propagation.
  * Added a preference "halfduplex" to force half-duplex communication
    with the server. This may be useful on unreliable links (as a more
    efficient alternative to "maxthreads =3D 1").
  * Renamed preference "pretendwin" to "ignoreinodenumbers" (an alias
    is kept for backwards compatibility).
  * Ignore one-second differences when synchronizing modification
    time. (Technically, this is an incompatible archive format change,
    but it is backward compatible. To trigger a problem, a user would
    have to synchronize modification times on a filesystem with a
    two-second granularity and then downgrade to a previous version of
    Unison, which does not work well in such a case. Thus, it does not
    seem worthwhile to increment the archive format number, which
    would impact all users.)
  * Do not keep many files simultaneously opened anymore when the
    rsync algorithm is in use.
  * Add "ignorearchives" preference to ignore existing archives (to
    avoid forcing users to delete them manually, in situations where
    one archive has gotten deleted or corrupted).
  * Added "BelowPath" patterns, that match a path as well as all paths
    below (convenient to use with nodeletion,update,creationpartial
    preferences)
  * Added a "fat" preference that makes Unison use the right options
    when one of the replicas is on a FAT filesystem.
  * Allow "prefer/force=3Dnewer" even when not synchronizing
    modification times. (The reconciler will not be aware of the
    modification time of unchanged files, so the synchronization
    choices of Unison can be different from when "times=3Dtrue", but the
    behavior remains sane: changed files with the most recent
    modification time will be propagated.)
  * Mac OS
       + fixed rsync bug which could result in an "index out of
         bounds" error when transferring resource forks.
       + Fixed bug which made Unison ignore finder information and
         resource fork when compiled to 64bit on Mac OSX.
       + should now be 64 bit clean (the Growl framework is not up to
         date, though)
       + Made the bridge between Objective C and Ocaml code GC
         friendly (it was allocating ML values and putting them in an
         array which was not registered with the GC)
       + use darker grey arrows (patch contributed by Eric Y. Kow)
  * GTK user interface
       + assistant for creating profiles
       + profile editor
       + pop up a summary window when the replicas are not fully
         synchronized after transport
       + display estimated remaining time and transfer rate on the
         progress bar
       + allow simultaneous selection of several items
       + Do not reload the preference file before a new update
         detection if it is unchanged
       + disabled scrolling to the first unfinished item during
         transport. It goes way too fast when lot of small files are
         synchronized, and it makes it impossible to browse the file
         list during transport.
       + take into account the "height" preference again
       + the internal list of selected reconciler item was not always
         in sync with what was displayed (GTK bug?); workaround
         implemented
       + Do not display "Looking for change" messages during
         propagation (when checking the targe is unchanged) but only
         during update detection
       + Apply patch to fix some crashes in the OSX GUI, thanks to
         Onne Gorter.
  * Text UI
       + During update detection, display status by updating a single
         line rather than generating a new line of output every so
         often. Should be less confusing.
  * Windows
       + Fastcheck is now the default under Windows. People mostly use
         NTFS nowadays and the Unicode API provides an equivalent to
         inode numbers for this filesystem.
       + Only use long UNC path for accessing replicas (as '..' is not
         handled with this format of paths, but can be useful)
       + Windows text UI: now put the console into UTF-8 output mode.
         This is the right thing to do when in Unicode mode, and is no
         worse than what we had previously otherwise (the console use
         some esoteric encoding by default). This only works when
         using a Unicode font instead of the default raster font.
       + Don't get the home directory from environment variable HOME
         under Windows (except for Cygwin binaries): we don't want the
         behavior of Unison to depends on whether it is run from a
         Cygwin shell (where HOME is set) or in any other way (where
         HOME is usually not set).
  * Miscellaneous fixes and improvements
       + Compare filenames up to decomposition in case sensitive mode
         when one host is running MacOSX and the unicode preference is
         set to true.
       + Rsync: somewhat faster compressor
       + Made a server waiting on a socket more resilient to
         unexpected lost connections from the client.
       + Small patch to property setting code suggested by Ulrich
         Gernkow.
       + Several fixes to the change transfer functions (both the
         internal ones and external transfers using rsync). In
         particular, limit the number of simultaneous transfer using
         an rsync (as the rsync algorithm can use a large amount of
         memory when processing huge files)
       + Keep track of which file contents are being transferred, and
         delay the transfer of a file when another file with the same
         contents is currently being transferred. This way, the second
         transfer can be skipped and replaced by a local copy.
       + Experimental update detection optimization: do not read the
         contents of unchanged directories
       + When a file transfer fails, turn off fastcheck for this file
         on the next sync.
       + Fixed bug with case insensitive mode on a case sensitive
         filesystem:
            o if file "a/a" is created on one replica and directory
              "A" is created on the other, the file failed to be
              synchronized the first time Unison is run afterwards, as
              Unison uses the wrong path "a/a" (if Unison is run
              again, the directories are in the archive, so the right
              path is used);
            o if file "a" appears on one replica and file "A" appears
              on the other with different contents, Unison was unable
              to synchronize them.
       + Improved error reporting when the destination is updated
         during synchronization: Unison now tells which file has been
         updated, and how.
       + Limit the length of temporary file names
       + Case sensitivity information put in the archive (in a
         backward compatible way) and checked when the archive is
         loaded
       + Got rid of the 16mb marshalling limit by marshalling to a
         bigarray.
       + Resume copy of partially transferred files.

Changes since 2.31:
  * Small user interface changes
       + Small change to text UI "scanning..." messages, to print just
         directories (hopefully making it clearer that individual
         files are not necessarily being fingerprinted).
  * Minor fixes and improvements:
       + Ignore one hour differences when deciding whether a file may
         have been updated. This avoids slow update detection after
         daylight saving time changes under Windows. This makes Unison
         slightly more likely to miss an update, but it should be safe
         enough.
       + Fix a small bug that was affecting mainly windows users. We
         need to commit the archives at the end of the sync even if
         there are no updates to propagate because some files (in
         fact, if we've just switched to DST on windows, a LOT of
         files) might have new modtimes in the archive. (Changed the
         text UI only. It's less clear where to change the GUI.)
       + Don't delete the temp file when a transfer fails due to a
         fingerprint mismatch (so that we can have a look and see
         why!) We've also added more debugging code togive more
         informative error messages when we encounter the dreaded and
         longstanding "assert failed during file transfer" bug
       + Incorrect paths ("path" directive) now result in an error
         update item rather than a fatal error.
       + Create parent directories (with correct permissions) during
         transport for paths which point to non-existent locations in
         the destination replica.

Changes since 2.27:
  * If Unison is interrupted during a directory transfer, it will now
    leave the partially transferred directory intact in a temporary
    location. (This maintains the invariant that new files/directories
    are transferred either completely or not at all.) The next time
    Unison is run, it will continue filling in this temporary
    directory, skipping transferring files that it finds are already
    there.
  * We've added experimental support for invoking an external file
    transfer tool for whole-file copies instead of Unison's built-in
    transfer protocol. Three new preferences have been added:
       + copyprog is a string giving the name (and command-line
         switches, if needed) of an external program that can be used
         to copy large files efficiently. By default, rsync is
         invoked, but other tools such as scp can be used instead by
         changing the value of this preference. (Although this is not
         its primary purpose, rsync is actually a pretty fast way of
         copying files that don't already exist on the receiving
         host.) For files that do already exist on (but that have been
         changed in one replica), Unison will always use its built-in
         implementation of the rsync algorithm.
       + Added a "copyprogrest" preference, so that we can give
         different command lines for invoking the external copy
         utility depending on whether a partially transferred file
         already exists or not. (Rsync doesn't seem to care about
         this, but other utilities may.)
       + copythreshold is an integer (-1 by default), indicating above
         what filesize (in megabytes) Unison should use the external
         copying utility specified by copyprog. Specifying 0 will
         cause ALL copies to use the external program; a negative
         number will prevent any files from using it. (Default is -1.)
    Thanks to Alan Schmitt for a huge amount of hacking and to an
    anonymous sponsor for suggesting and underwriting this extension.
  * Small improvements:
       + Added a new preference, dontchmod. By default, Unison uses
         the chmod system call to set the permission bits of files
         after it has copied them. But in some circumstances (and
         under some operating systems), the chmod call always fails.
         Setting this preference completely prevents Unison from ever
         calling chmod.
       + Don't ignore files that look like backup files if the
         backuplocation preference is set to central
       + Shortened the names of several preferences. The old names are
         also still supported, for backwards compatibility, but they
         do not appear in the documentation.
       + Lots of little documentation tidying. (In particular,
         preferences are separated into Basic and Advanced! This
         should hopefully make Unison a little more approachable for
         new users.
       + Unison can sometimes fail to transfer a file, giving the
         unhelpful message "Destination updated during
         synchronization" even though the file has not been changed.
         This can be caused by programs that change either the file's
         contents or the file's extended attributes without changing
         its modification time. It's not clear what is the best fix
         for this - it is not Unison's fault, but it makes Unison's
         behavior puzzling - but at least Unison can be more helpful
         about suggesting a workaround (running once with fastcheck
         set to false). The failure message has been changed to give
         this advice.
       + Further improvements to the OS X GUI (thanks to Alan Schmitt
         and Craig Federighi).
  * Very preliminary support for triggering Unison from an external
    filesystem-watching utility. The current implementation is very
    simple, not efficient, and almost completely untested--not ready
    for real users. But if someone wants to help improve it (e.g., by
    writing a filesystem watcher for your favorite OS), please make
    yourself known!
    On the Unison side, the new behavior is very simple:
       + use the text UI
       + start Unison with the command-line flag "-repeat FOO", where
         FOO is name of a file where Unison should look for
         notifications of changes
       + when it starts up, Unison will read the whole contents of
         this file (on both hosts), which should be a
         newline-separated list of paths (relative to the root of the
         synchronization) and synchronize just these paths, as if it
         had been started with the "-path=3Dxxx" option for each one of
         them
       + when it finishes, it will sleep for a few seconds and then
         examine the watchfile again; if anything has been added, it
         will read the new paths, synchronize them, and go back to
         sleep
       + that's it!
    To use this to drive Unison "incrementally," just start it in this
    mode and start up a tool (on each host) to watch for new changes
    to the filesystem and append the appropriate paths to the
    watchfile. Hopefully such tools should not be too hard to write.
  * Bug fixes:
       + Fixed a bug that was causing new files to be created with
         permissions 0x600 instead of using a reasonable default (like
         0x644), if the 'perms' flag was set to 0. (Bug reported by
         Ben Crowell.)
       + Follow maxthreads preference when transferring directories.


=20=20=20

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected]=20
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/