Loading ...
Sorry, an error occurred while loading the content.

Re: [nslu2-general] Re: Swap partition

Expand Messages
  • Fajar A. Nugraha
    ... Definitely. Better NOT use swap file on some cases. If your program needs more memory, then you can t run it on your plug. Full-blown linux distro have
    Message 1 of 10 , Feb 29, 2012
    • 0 Attachment
      On Thu, Mar 1, 2012 at 10:35 AM, adamant628 <adam.suhy@...> wrote:
      > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?

      Definitely.

      Better NOT use swap file on some cases. If your program needs more
      memory, then you can't run it on your plug.

      Full-blown linux distro have zram, which can take part of main memory,
      compress it, then use it as swap. In total it gives "more" usable
      memory to programs. Don't expect anything too drastic though, you'd
      probably only get about 50% more usabe memory.

      >
      > So how does it work if the swapfile is larger than the actual memory?

      If you have many programs loaded to memory, but only several of them
      are active, the "working data" (i.e. anonymous memory) of the inactive
      program can be put in swap to make room for active program, while
      memory used by executables (i.e. "programs" loaded from disk) can
      either be swapped, or removed (since they can be loaded later by
      re-reading the files).

      If your currently-inactive program becomes active, or if you use one
      program that needs more than available memory, your system will be too
      busy reading and writing to-and-from swap to be doing anything else.
      It won't be usable.

      So in short, if you're running only ONE program, and it complains not
      having enough memory, most likely you should make it use less, or
      don't use it. Don't resort to adding swap just to get that program
      running.

      This page should be informative:
      http://oreilly.com/catalog/spt2/chapter/ch04.html#79188

      --
      Fajar
    • adamant628
      Thanks for the information and the reading. Perhaps swap is not what I need, and I have another problem. I ran top to keep an eye on the memory usage, and I
      Message 2 of 10 , Feb 29, 2012
      • 0 Attachment
        Thanks for the information and the reading. Perhaps swap is not what I need, and I have another problem.

        I ran top to keep an eye on the memory usage, and I am seeing something odd:

        Here is the top line:

        Mem: 248640K used, 7368K free, 0K shrd, 126832K buff, 43596K cached

        I added up the VSZ of everything running and it only comes to 47272.

        How can I figure out what is using up all of that memory? I don't have any programs running other than the default pogoplug stuff (hbplug using the majority ~25MB).

        Is it normal to have so little free memory?

        --- In nslu2-general@yahoogroups.com, "Fajar A. Nugraha" <fajar@...> wrote:
        >
        > On Thu, Mar 1, 2012 at 10:35 AM, adamant628 <adam.suhy@...> wrote:
        > > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?
        >
        > Definitely.
        >
        > Better NOT use swap file on some cases. If your program needs more
        > memory, then you can't run it on your plug.
        >
        > Full-blown linux distro have zram, which can take part of main memory,
        > compress it, then use it as swap. In total it gives "more" usable
        > memory to programs. Don't expect anything too drastic though, you'd
        > probably only get about 50% more usabe memory.
        >
        > >
        > > So how does it work if the swapfile is larger than the actual memory?
        >
        > If you have many programs loaded to memory, but only several of them
        > are active, the "working data" (i.e. anonymous memory) of the inactive
        > program can be put in swap to make room for active program, while
        > memory used by executables (i.e. "programs" loaded from disk) can
        > either be swapped, or removed (since they can be loaded later by
        > re-reading the files).
        >
        > If your currently-inactive program becomes active, or if you use one
        > program that needs more than available memory, your system will be too
        > busy reading and writing to-and-from swap to be doing anything else.
        > It won't be usable.
        >
        > So in short, if you're running only ONE program, and it complains not
        > having enough memory, most likely you should make it use less, or
        > don't use it. Don't resort to adding swap just to get that program
        > running.
        >
        > This page should be informative:
        > http://oreilly.com/catalog/spt2/chapter/ch04.html#79188
        >
        > --
        > Fajar
        >
      • Fajar A. Nugraha
        ... Yes, if all are used by buffer and cache I like this two commands: (1) ps -eo %cpu,%mem,vsz,rss,args --sort rss you need full version of ps (e.g. from
        Message 3 of 10 , Feb 29, 2012
        • 0 Attachment
          On Thu, Mar 1, 2012 at 12:28 PM, adamant628 <adam.suhy@...> wrote:
          > Thanks for the information and the reading.  Perhaps swap is not what I need, and I have another problem.
          >
          > I ran top to keep an eye on the memory usage, and I am seeing something odd:
          >
          > Here is the top line:
          >
          > Mem: 248640K used, 7368K free, 0K shrd, 126832K buff, 43596K cached
          >
          > I added up the VSZ of everything running and it only comes to 47272.
          >
          > How can I figure out what is using up all of that memory?  I don't have any programs running other than the default pogoplug stuff (hbplug using the majority ~25MB).
          >
          > Is it normal to have so little free memory?

          Yes, if all are used by buffer and cache

          I like this two commands:
          (1) ps -eo %cpu,%mem,vsz,rss,args --sort rss
          you need full version of ps (e.g. from optware, not the one from busybox)
          (2) htop
          The colors really help. For memory:
          green -> used
          yellow -> cache
          blue -> buffers

          the memory used by buffer and cache should be usable by other programs
          as needed, you don't have to do anything.

          --
          Fajar
        • Bill
          A swap file simply acts as extra memory blocks. Lets take a very simple case. Lets say you had six memory blocks, and each program took one memory block
          Message 4 of 10 , Mar 1, 2012
          • 0 Attachment
            A swap file simply acts as extra memory blocks. Lets take a very simple case. Lets say you had six memory blocks, and each program took one memory block each while running. Then if you were to run 18 programs, at any given time 12 would be swapped to disk instead of being available to actively run. It turns out often times problems are just waiting for IO response some other event, and many of the memory blocks they allocate are not referenced that frequently, so this kind of ratio works out well for some system loads. I found on my NSLU2 I would typically run with 2 to 3 times the amount of swap in use as actual memory. Some programs such as apache took more memory than the NSLU2 had just to start-up. But so long as the actual system load (usage) was light it worked out fine. When I would run into problem is when I tried to stream a video via the apache web server, copy files via samba, and download e-mail with dovecot all at the same time. Then the NSLU2 would grind to a halt because it was spending all its time reading and writing memory blocks from disk.

            As for putting swap on an flash device, it is a bad idea. As it will shorten the life of the drive if you use the swap frequently. If you have a swap on disk and one on flash generally it works out fine to just make the flash swap lower priority. Another option is just to use a very inexpensive flash device you don't mind replacing when needed.

            Since I haven't really looked at the directions I'll just give my own here:

            dd if=/dev/zero of=/swap.img bs=1M count=256
            mkswap -L myswap /swap.img
            swapon /swap.img

            To verify:

            swapon -s

            or:

            free
          • Fajar A. Nugraha
            ... IMHO if you had programs like that, it s a sign of bad configuration. You should be able to easily reduce memory usage for a web server by (for example): -
            Message 5 of 10 , Mar 1, 2012
            • 0 Attachment
              On Fri, Mar 2, 2012 at 9:55 AM, Bill <yahoo@...> wrote:
              > Some programs such as apache took more memory than the NSLU2 had just to start-up.

              IMHO if you had programs like that, it's a sign of bad configuration.

              You should be able to easily reduce memory usage for a web server by
              (for example):
              - changing from mpm_prefok to mpm_worker/mpm_event, and if you use php
              also change it to use fcgi + php_fpm.
              - change any program that spawns multiple process (e.g. apache with
              mpm_prefork, php_fpm, etc) to use as little process as possible. For
              example, php_fpm should run fine with just 1 process max, while for
              apache you could probably limit it to 4 (or switch to worker/event).
              It's a for-personal-use embedded device anyway, not like you're going
              to have several hundred users accessing it at once.
              - switch to a more memory-efficient server (e.g. lighttpd)

              > But so long as the actual system load (usage) was light it worked out fine

              True.

              But if it were me I'd rather limit the max amount of memory a program
              is going to use in it's lifetime. And if there's higher load (e.g.
              many users accessing the web server) I want them to wait in queue, or
              outright rejected. I don't want it to suddenly grind to a halt due to
              swapping.

              --
              Fajar
            • Bill
              Sorry I should have been more specific /swap.img is just a place holder value. You need to actually create it somewhere that is externally mounted. I do not
              Message 6 of 10 , Mar 2, 2012
              • 0 Attachment
                Sorry I should have been more specific /swap.img is just a place holder value. You need to actually create it somewhere that is externally mounted. I do not know if you are using SlugOS, or Optware with a custom firmware, Debian. With Debian /swap.img will work, because it won't be an in-memory device. If you / is a just a memory mount then it won't work, because linux is smart enough not to let you use a ramdisk or such as swapspace.

                Bill

                --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@...> wrote:
                >
                > I did use the mkswap command, just as noted in the how-to. I just tried again and got the same error message.
                >
                > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?
                >
                > So how does it work if the swapfile is larger than the actual memory?
                >
                > --- In nslu2-general@yahoogroups.com, "Bill" <yahoo@> wrote:
                > >
                > > You can make your swap file any size you want. (Within reason.) A small amount of non-swappable memory is used to map data in the swap file, so I would not recommend much more than 8 times the actual amount of memory you have on the device. Most people say 2 times the actual memory is a good size to use.
                > >
                > > Did you remember to use the mkswap command before activating your swap?
                > >
                > > Bill
                > >
                > >
                > > --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@> wrote:
                > > >
                > > > I am pretty sure that I need to create some swap space, as one of the programs I am running is complaining on not having enough memory. I was following the instructions on http://www.nslu2-linux.org/wiki/HowTo/AddAdditionalSwapSpace to add a swap file. I am a linux newbie, but I have been learning. I successfully have programs running on my optware enabled pogoplug.
                > > >
                > > >
                > > > Regarding the link above, the command: "dd if=/dev/zero of=/mnt/swapfile1 bs=1024 count=256k" is creating the swap file from the flash memory of the device? The how-to mentions that this will create a 256 MB swap file. I only have 125MB available on my device, so I assume that I shouldn't exceed that amount.
                > > >
                > > > I attempted to follow these instructions (made my swapfile 64MB), but when I used "swapon" I was given an error: "swapon failed: Invalid argument". This stumped me, so I don't know how to proceed at this point.
                > > >
                > > >
                > > > My basic understanding of swap is that it is basically the same as virtual memory in windows. I have also read that it can burn out a USB flash drive fairly quickly due to the increased read/write cycles. Is it safe to do this on the pogoplug itself, or should I dedicate a portion of one of my externals for the task? I assume that if I use an external, I would skip the part about making the file, then direct mkswap to that partition's mount?
                > > >
                > > > Thanks for your help.
                > > >
                > >
                >
              • Bill
                ... I agree. I used apache as a specific example, because it was one for awhile I had no choice to use. I had all the memory reducing options set at the
                Message 7 of 10 , Mar 2, 2012
                • 0 Attachment
                  --- In nslu2-general@yahoogroups.com, "Fajar A. Nugraha" <fajar@...> wrote:
                  >
                  > On Fri, Mar 2, 2012 at 9:55 AM, Bill <yahoo@...> wrote:
                  > > Some programs such as apache took more memory than the NSLU2 had just to start-up.
                  >
                  > IMHO if you had programs like that, it's a sign of bad configuration.

                  I agree. I used apache as a specific example, because it was one for awhile I had no choice to use. I had all the memory reducing options set at the time, but during startup apache would still use more memory than an NSLU2 has available. Once started though, apache would drop it's memory usage to more reasonable.

                  Eventually, the bugs with lighttpd that prevented it from being usable for streaming videos were fixed and I switched to that. Although once running both lighttpd and apache use about the same amount of memory...

                  Bill
                Your message has been successfully submitted and would be delivered to recipients shortly.