Loading ...
Sorry, an error occurred while loading the content.

Re: [unison-users] Unison hangs indefinitely

Expand Messages
  • Jerome Vouillon
    ... If you have a direct ethernet connection, it is unlikely that you observe any timeout. On the other hand, the initial synchronization of two replicas with
    Message 1 of 11 , Oct 5, 2005
    • 0 Attachment
      On Sun, Oct 02, 2005 at 02:32:46PM -0400, Chuck Farley wrote:
      > I will be mirroring full 1 gig files regularly. I
      > will however have a direct 1Gbps ethernet connection
      > on a peer to peer type basis for the sync.

      If you have a direct ethernet connection, it is unlikely that you
      observe any timeout.

      On the other hand, the initial synchronization of two replicas with a
      large number of files can fail: the server tries to send the
      information about all the files back to the client in one go, but
      there is currently a 16 megabytes limitation on what Unison can send
      in one go. The workaround is to use the "path" directive to
      synchronize smaller pieces of the replicas at a time. This is only an
      issue for the initial synchronization: afterwards, the server just
      sends some information about which files have been modified, which is
      normally quite small.

      -- Jerome
    • Chuck Farley
      Hrm...Interesting...Well I broke my job into multiple paths as suggested. I synced up a few paths no problems then added a whammy of a directory that was
      Message 2 of 11 , Oct 5, 2005
      • 0 Attachment
        Hrm...Interesting...Well I broke my job into multiple
        paths as suggested. I synced up a few paths no
        problems then added a whammy of a directory that was
        29Gigs in size. Everything went fine until the end:

        UNISON started propagating changes at 11:24:23 on 05
        Oct 2005
        [BGN] Copying uwstout
        from /ep/epgroup
        to //dale2//ep/epgroup
        99% 00:00 ETA

        It has been sitting on 99% for about 20 minutes now.
        I'm wondering though if maybe it is going through and
        setting perms and time stamps on the files. Since I
        know there are a NUMBER of files in this particular
        dir it could take some time if it does them all after
        the fact.

        I'm scared to terminate the process as I believe it
        may still be doing things (top on the server side
        shows memory size changing and procesor usage).

        When you say 16 megs in size is that file size? So
        Unison will have problems syncing a file larger than
        16 megs?

        --- Jerome Vouillon <Jerome.Vouillon@...>
        wrote:

        > On Sun, Oct 02, 2005 at 02:32:46PM -0400, Chuck
        > Farley wrote:
        > > I will be mirroring full 1 gig files regularly. I
        > > will however have a direct 1Gbps ethernet
        > connection
        > > on a peer to peer type basis for the sync.
        >
        > If you have a direct ethernet connection, it is
        > unlikely that you
        > observe any timeout.
        >
        > On the other hand, the initial synchronization of
        > two replicas with a
        > large number of files can fail: the server tries to
        > send the
        > information about all the files back to the client
        > in one go, but
        > there is currently a 16 megabytes limitation on what
        > Unison can send
        > in one go. The workaround is to use the "path"
        > directive to
        > synchronize smaller pieces of the replicas at a
        > time. This is only an
        > issue for the initial synchronization: afterwards,
        > the server just
        > sends some information about which files have been
        > modified, which is
        > normally quite small.
        >
        > -- Jerome
        >







        __________________________________________________________
        Find your next car at http://autos.yahoo.ca
      • Chuck Farley
        Scratch my last e-mail...Looks like I was correct in that it was still running...Just took a long time to finish up for some reason. So that was a 29 Gig
        Message 3 of 11 , Oct 5, 2005
        • 0 Attachment
          Scratch my last e-mail...Looks like I was correct in
          that it was still running...Just took a long time to
          finish up for some reason.

          So that was a 29 Gig directory tree. Synced in about
          2 hours with no issue (aside from Unison not being
          accurate with its ETA.. :)

          Quick question though: I would like to sync
          timestamps. I have times = true in default.prf and
          it's correcting the time stamps of files but not
          directories. When I don't use times = true neither
          are corrected. Did I miss something in the docs?


          --- Jerome Vouillon <Jerome.Vouillon@...>
          wrote:

          > On Sun, Oct 02, 2005 at 02:32:46PM -0400, Chuck
          > Farley wrote:
          > > I will be mirroring full 1 gig files regularly. I
          > > will however have a direct 1Gbps ethernet
          > connection
          > > on a peer to peer type basis for the sync.
          >
          > If you have a direct ethernet connection, it is
          > unlikely that you
          > observe any timeout.
          >
          > On the other hand, the initial synchronization of
          > two replicas with a
          > large number of files can fail: the server tries to
          > send the
          > information about all the files back to the client
          > in one go, but
          > there is currently a 16 megabytes limitation on what
          > Unison can send
          > in one go. The workaround is to use the "path"
          > directive to
          > synchronize smaller pieces of the replicas at a
          > time. This is only an
          > issue for the initial synchronization: afterwards,
          > the server just
          > sends some information about which files have been
          > modified, which is
          > normally quite small.
          >
          > -- Jerome
          >







          __________________________________________________________
          Find your next car at http://autos.yahoo.ca
        • Drew McDermott
          ... My long response time illustrates the dangers of letting one s e-mail get too backlogged. I ve actually been making do with a shell script that
          Message 4 of 11 , Oct 30, 2005
          • 0 Attachment
            > [Jerome Vouillon]
            > On Fri, Sep 30, 2005 at 09:19:32PM -0400, Drew McDermott wrote:
            > >
            > > I have relied on Unison for a couple of years now, but for the last
            > > few months its behavior has been driving me crazy. When trying to
            > > synchronize a directory more than about 1 GB in size, it often hangs
            > > indefinitely after printing "Waiting for changes from server." I can
            > > synchronize all the subdirectories separately, and that sometimes
            > > fixes the problem, but not always.
            > >
            > > The thing is, it used to work just fine, and the directory in question
            > > grew significantly above 1GB before the problem hit. It first
            > > manifested itself on Windows XP machines, but now it seems to affect
            > > Unix machines (Linux and Mac OS X) as well.
            >
            > This is probably because the connection between the client and the
            > server is idle for too long. Then, a timeout is triggered at a
            > masquerading router or something similar.
            >
            > If you are using ssh, can you try to run Unison with the sshargs
            > directive set to "-o ServerAliveInterval=60": either run unison as
            > ./unison -sshargs "-o ServerAliveInterval=60" <other-options>
            > or add
            > sshargs = -o ServerAliveInterval=60
            > in the Unison profile.
            >
            > Another option is to use run Unison through a tunnel such as OpenVPN
            > <http://openvpn.net/>.
            >
            > Please tell us whether any of these work for you.
            >

            My long response time illustrates the dangers of letting one's e-mail
            get too backlogged.

            I've actually been making do with a shell script that synchronizes
            five subdirectories of my overall directory. That is, I've been
            making do on my Mac; on my Windows machine the profile for the big
            directory worked fine.

            Then I read your message, and tried the sshargs thing, and it seems to
            have worked. Although the shell script wasn't too much of an
            inconvenience (I had at most 5 different q-and-a sessions with Unison
            asking me for what to do with files), working directly with the
            superdirectory is certainly better.

            Thanks for all the suggestions.

            -- Drew


            --

            -- Drew McDermott
            Yale University
            Computer Science Department

            Source close to the White House have said that, after the indictment
            of Scooter Libby, the Vice President appears to be in such serious
            trouble that President Bush has offered to resign.
          Your message has been successfully submitted and would be delivered to recipients shortly.