Loading ...
Sorry, an error occurred while loading the content.
 

Re: Bug#579213: Please allow an option to posix_fadvise files read not to be cached

Expand Messages
  • Sylvain Le Gall
    Hello, ... I played a bit with fadvise and I even have an OCaml binding for it, somewhere on my hard-drive. I think it can be quite efficient but you should
    Message 1 of 2 , Apr 26, 2010
      Hello,

      On Mon, Apr 26, 2010 at 10:17:49AM +0100, Enrico Zini wrote:
      > My laptop has 2GB of RAM, but sometimes I want to sync directory trees
      > with a similar order of magnitude of data in them. When unison scans the
      > local data, linux has a tendency to swap out running application to free
      > RAM for caching the data that unison reads. The result is that while
      > unison is running, my laptop is thrashing heavily as the kernel loads
      > back from swap those apps that it just swapped.
      >
      > I'm running with default kernel VM settings. Running unison under
      > nocache (http://www.enricozini.org/2010/tips/nocache/) does not cause
      > the thrashing and keeps the system perfectly usable during syncs, if a
      > bit slower I/O wise, which is perfectly reasonable.
      >
      > I am of course the first to agree that the problem in this is not in
      > unison but in Linux itself. The operating system should not swap out
      > running apps: this behaviour is preposterous, and everytime I experience
      > it a part of my soul dies drowned by its own tears.
      >
      > It might still make sense, however, to have an option in unison
      > (disabled by default) that turns on fadvise cache hinting. One would
      > turn it on when syncing big, seldom accessed archives like photo
      > galleries or music collections, and keep it off when syncing the home
      > dir with files that are accessed often.
      >
      > Even more ideal would be to be able to turn it on/off on a per-directory
      > basis, so that I can sync my home dir and have the system cache
      > everything except pictures, music and videos.
      >

      I played a bit with fadvise and I even have an OCaml binding for it,
      somewhere on my hard-drive. I think it can be quite efficient but you
      should talk about this directly with upstream.

      I forward this mail to upstream, so that you can discuss it.

      The original bug is here:
      http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=579213

      Regards
      Sylvain Le Gall
    • Benjamin Pierce
      Sounds like a refinement that would be useful to quite a few people. If someone cares to implement it, I d be glad to consider a patch. Main issue is going
      Message 2 of 2 , Apr 26, 2010
        Sounds like a refinement that would be useful to quite a few people. If someone cares to implement it, I'd be glad to consider a patch. Main issue is going to be making sure it compiles on all platforms and does something reasonable on as many as possible.

        - Benjamin


        On Apr 26, 2010, at 5:45 AM, Sylvain Le Gall wrote:

        > Hello,
        >
        > On Mon, Apr 26, 2010 at 10:17:49AM +0100, Enrico Zini wrote:
        >> My laptop has 2GB of RAM, but sometimes I want to sync directory trees
        >> with a similar order of magnitude of data in them. When unison scans the
        >> local data, linux has a tendency to swap out running application to free
        >> RAM for caching the data that unison reads. The result is that while
        >> unison is running, my laptop is thrashing heavily as the kernel loads
        >> back from swap those apps that it just swapped.
        >>
        >> I'm running with default kernel VM settings. Running unison under
        >> nocache (http://www.enricozini.org/2010/tips/nocache/) does not cause
        >> the thrashing and keeps the system perfectly usable during syncs, if a
        >> bit slower I/O wise, which is perfectly reasonable.
        >>
        >> I am of course the first to agree that the problem in this is not in
        >> unison but in Linux itself. The operating system should not swap out
        >> running apps: this behaviour is preposterous, and everytime I experience
        >> it a part of my soul dies drowned by its own tears.
        >>
        >> It might still make sense, however, to have an option in unison
        >> (disabled by default) that turns on fadvise cache hinting. One would
        >> turn it on when syncing big, seldom accessed archives like photo
        >> galleries or music collections, and keep it off when syncing the home
        >> dir with files that are accessed often.
        >>
        >> Even more ideal would be to be able to turn it on/off on a per-directory
        >> basis, so that I can sync my home dir and have the system cache
        >> everything except pictures, music and videos.
        >>
        >
        > I played a bit with fadvise and I even have an OCaml binding for it,
        > somewhere on my hard-drive. I think it can be quite efficient but you
        > should talk about this directly with upstream.
        >
        > I forward this mail to upstream, so that you can discuss it.
        >
        > The original bug is here:
        > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=579213
        >
        > Regards
        > Sylvain Le Gall
        >
        >
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.