Loading ...
Sorry, an error occurred while loading the content.

Re: Swap partition

Expand Messages
  • Bill
    You can make your swap file any size you want. (Within reason.) A small amount of non-swappable memory is used to map data in the swap file, so I would not
    Message 1 of 10 , Feb 29, 2012
    • 0 Attachment
      You can make your swap file any size you want. (Within reason.) A small amount of non-swappable memory is used to map data in the swap file, so I would not recommend much more than 8 times the actual amount of memory you have on the device. Most people say 2 times the actual memory is a good size to use.

      Did you remember to use the mkswap command before activating your swap?

      Bill


      --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@...> wrote:
      >
      > I am pretty sure that I need to create some swap space, as one of the programs I am running is complaining on not having enough memory. I was following the instructions on http://www.nslu2-linux.org/wiki/HowTo/AddAdditionalSwapSpace to add a swap file. I am a linux newbie, but I have been learning. I successfully have programs running on my optware enabled pogoplug.
      >
      >
      > Regarding the link above, the command: "dd if=/dev/zero of=/mnt/swapfile1 bs=1024 count=256k" is creating the swap file from the flash memory of the device? The how-to mentions that this will create a 256 MB swap file. I only have 125MB available on my device, so I assume that I shouldn't exceed that amount.
      >
      > I attempted to follow these instructions (made my swapfile 64MB), but when I used "swapon" I was given an error: "swapon failed: Invalid argument". This stumped me, so I don't know how to proceed at this point.
      >
      >
      > My basic understanding of swap is that it is basically the same as virtual memory in windows. I have also read that it can burn out a USB flash drive fairly quickly due to the increased read/write cycles. Is it safe to do this on the pogoplug itself, or should I dedicate a portion of one of my externals for the task? I assume that if I use an external, I would skip the part about making the file, then direct mkswap to that partition's mount?
      >
      > Thanks for your help.
      >
    • adamant628
      I did use the mkswap command, just as noted in the how-to. I just tried again and got the same error message. Can you also address my other concern about the
      Message 2 of 10 , Feb 29, 2012
      • 0 Attachment
        I did use the mkswap command, just as noted in the how-to. I just tried again and got the same error message.

        Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?

        So how does it work if the swapfile is larger than the actual memory?

        --- In nslu2-general@yahoogroups.com, "Bill" <yahoo@...> wrote:
        >
        > You can make your swap file any size you want. (Within reason.) A small amount of non-swappable memory is used to map data in the swap file, so I would not recommend much more than 8 times the actual amount of memory you have on the device. Most people say 2 times the actual memory is a good size to use.
        >
        > Did you remember to use the mkswap command before activating your swap?
        >
        > Bill
        >
        >
        > --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@> wrote:
        > >
        > > I am pretty sure that I need to create some swap space, as one of the programs I am running is complaining on not having enough memory. I was following the instructions on http://www.nslu2-linux.org/wiki/HowTo/AddAdditionalSwapSpace to add a swap file. I am a linux newbie, but I have been learning. I successfully have programs running on my optware enabled pogoplug.
        > >
        > >
        > > Regarding the link above, the command: "dd if=/dev/zero of=/mnt/swapfile1 bs=1024 count=256k" is creating the swap file from the flash memory of the device? The how-to mentions that this will create a 256 MB swap file. I only have 125MB available on my device, so I assume that I shouldn't exceed that amount.
        > >
        > > I attempted to follow these instructions (made my swapfile 64MB), but when I used "swapon" I was given an error: "swapon failed: Invalid argument". This stumped me, so I don't know how to proceed at this point.
        > >
        > >
        > > My basic understanding of swap is that it is basically the same as virtual memory in windows. I have also read that it can burn out a USB flash drive fairly quickly due to the increased read/write cycles. Is it safe to do this on the pogoplug itself, or should I dedicate a portion of one of my externals for the task? I assume that if I use an external, I would skip the part about making the file, then direct mkswap to that partition's mount?
        > >
        > > Thanks for your help.
        > >
        >
      • Fajar A. Nugraha
        ... Definitely. Better NOT use swap file on some cases. If your program needs more memory, then you can t run it on your plug. Full-blown linux distro have
        Message 3 of 10 , Feb 29, 2012
        • 0 Attachment
          On Thu, Mar 1, 2012 at 10:35 AM, adamant628 <adam.suhy@...> wrote:
          > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?

          Definitely.

          Better NOT use swap file on some cases. If your program needs more
          memory, then you can't run it on your plug.

          Full-blown linux distro have zram, which can take part of main memory,
          compress it, then use it as swap. In total it gives "more" usable
          memory to programs. Don't expect anything too drastic though, you'd
          probably only get about 50% more usabe memory.

          >
          > So how does it work if the swapfile is larger than the actual memory?

          If you have many programs loaded to memory, but only several of them
          are active, the "working data" (i.e. anonymous memory) of the inactive
          program can be put in swap to make room for active program, while
          memory used by executables (i.e. "programs" loaded from disk) can
          either be swapped, or removed (since they can be loaded later by
          re-reading the files).

          If your currently-inactive program becomes active, or if you use one
          program that needs more than available memory, your system will be too
          busy reading and writing to-and-from swap to be doing anything else.
          It won't be usable.

          So in short, if you're running only ONE program, and it complains not
          having enough memory, most likely you should make it use less, or
          don't use it. Don't resort to adding swap just to get that program
          running.

          This page should be informative:
          http://oreilly.com/catalog/spt2/chapter/ch04.html#79188

          --
          Fajar
        • adamant628
          Thanks for the information and the reading. Perhaps swap is not what I need, and I have another problem. I ran top to keep an eye on the memory usage, and I
          Message 4 of 10 , Feb 29, 2012
          • 0 Attachment
            Thanks for the information and the reading. Perhaps swap is not what I need, and I have another problem.

            I ran top to keep an eye on the memory usage, and I am seeing something odd:

            Here is the top line:

            Mem: 248640K used, 7368K free, 0K shrd, 126832K buff, 43596K cached

            I added up the VSZ of everything running and it only comes to 47272.

            How can I figure out what is using up all of that memory? I don't have any programs running other than the default pogoplug stuff (hbplug using the majority ~25MB).

            Is it normal to have so little free memory?

            --- In nslu2-general@yahoogroups.com, "Fajar A. Nugraha" <fajar@...> wrote:
            >
            > On Thu, Mar 1, 2012 at 10:35 AM, adamant628 <adam.suhy@...> wrote:
            > > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?
            >
            > Definitely.
            >
            > Better NOT use swap file on some cases. If your program needs more
            > memory, then you can't run it on your plug.
            >
            > Full-blown linux distro have zram, which can take part of main memory,
            > compress it, then use it as swap. In total it gives "more" usable
            > memory to programs. Don't expect anything too drastic though, you'd
            > probably only get about 50% more usabe memory.
            >
            > >
            > > So how does it work if the swapfile is larger than the actual memory?
            >
            > If you have many programs loaded to memory, but only several of them
            > are active, the "working data" (i.e. anonymous memory) of the inactive
            > program can be put in swap to make room for active program, while
            > memory used by executables (i.e. "programs" loaded from disk) can
            > either be swapped, or removed (since they can be loaded later by
            > re-reading the files).
            >
            > If your currently-inactive program becomes active, or if you use one
            > program that needs more than available memory, your system will be too
            > busy reading and writing to-and-from swap to be doing anything else.
            > It won't be usable.
            >
            > So in short, if you're running only ONE program, and it complains not
            > having enough memory, most likely you should make it use less, or
            > don't use it. Don't resort to adding swap just to get that program
            > running.
            >
            > This page should be informative:
            > http://oreilly.com/catalog/spt2/chapter/ch04.html#79188
            >
            > --
            > Fajar
            >
          • Fajar A. Nugraha
            ... Yes, if all are used by buffer and cache I like this two commands: (1) ps -eo %cpu,%mem,vsz,rss,args --sort rss you need full version of ps (e.g. from
            Message 5 of 10 , Feb 29, 2012
            • 0 Attachment
              On Thu, Mar 1, 2012 at 12:28 PM, adamant628 <adam.suhy@...> wrote:
              > Thanks for the information and the reading.  Perhaps swap is not what I need, and I have another problem.
              >
              > I ran top to keep an eye on the memory usage, and I am seeing something odd:
              >
              > Here is the top line:
              >
              > Mem: 248640K used, 7368K free, 0K shrd, 126832K buff, 43596K cached
              >
              > I added up the VSZ of everything running and it only comes to 47272.
              >
              > How can I figure out what is using up all of that memory?  I don't have any programs running other than the default pogoplug stuff (hbplug using the majority ~25MB).
              >
              > Is it normal to have so little free memory?

              Yes, if all are used by buffer and cache

              I like this two commands:
              (1) ps -eo %cpu,%mem,vsz,rss,args --sort rss
              you need full version of ps (e.g. from optware, not the one from busybox)
              (2) htop
              The colors really help. For memory:
              green -> used
              yellow -> cache
              blue -> buffers

              the memory used by buffer and cache should be usable by other programs
              as needed, you don't have to do anything.

              --
              Fajar
            • Bill
              A swap file simply acts as extra memory blocks. Lets take a very simple case. Lets say you had six memory blocks, and each program took one memory block
              Message 6 of 10 , Mar 1 6:55 PM
              • 0 Attachment
                A swap file simply acts as extra memory blocks. Lets take a very simple case. Lets say you had six memory blocks, and each program took one memory block each while running. Then if you were to run 18 programs, at any given time 12 would be swapped to disk instead of being available to actively run. It turns out often times problems are just waiting for IO response some other event, and many of the memory blocks they allocate are not referenced that frequently, so this kind of ratio works out well for some system loads. I found on my NSLU2 I would typically run with 2 to 3 times the amount of swap in use as actual memory. Some programs such as apache took more memory than the NSLU2 had just to start-up. But so long as the actual system load (usage) was light it worked out fine. When I would run into problem is when I tried to stream a video via the apache web server, copy files via samba, and download e-mail with dovecot all at the same time. Then the NSLU2 would grind to a halt because it was spending all its time reading and writing memory blocks from disk.

                As for putting swap on an flash device, it is a bad idea. As it will shorten the life of the drive if you use the swap frequently. If you have a swap on disk and one on flash generally it works out fine to just make the flash swap lower priority. Another option is just to use a very inexpensive flash device you don't mind replacing when needed.

                Since I haven't really looked at the directions I'll just give my own here:

                dd if=/dev/zero of=/swap.img bs=1M count=256
                mkswap -L myswap /swap.img
                swapon /swap.img

                To verify:

                swapon -s

                or:

                free
              • Fajar A. Nugraha
                ... IMHO if you had programs like that, it s a sign of bad configuration. You should be able to easily reduce memory usage for a web server by (for example): -
                Message 7 of 10 , Mar 1 7:56 PM
                • 0 Attachment
                  On Fri, Mar 2, 2012 at 9:55 AM, Bill <yahoo@...> wrote:
                  > Some programs such as apache took more memory than the NSLU2 had just to start-up.

                  IMHO if you had programs like that, it's a sign of bad configuration.

                  You should be able to easily reduce memory usage for a web server by
                  (for example):
                  - changing from mpm_prefok to mpm_worker/mpm_event, and if you use php
                  also change it to use fcgi + php_fpm.
                  - change any program that spawns multiple process (e.g. apache with
                  mpm_prefork, php_fpm, etc) to use as little process as possible. For
                  example, php_fpm should run fine with just 1 process max, while for
                  apache you could probably limit it to 4 (or switch to worker/event).
                  It's a for-personal-use embedded device anyway, not like you're going
                  to have several hundred users accessing it at once.
                  - switch to a more memory-efficient server (e.g. lighttpd)

                  > But so long as the actual system load (usage) was light it worked out fine

                  True.

                  But if it were me I'd rather limit the max amount of memory a program
                  is going to use in it's lifetime. And if there's higher load (e.g.
                  many users accessing the web server) I want them to wait in queue, or
                  outright rejected. I don't want it to suddenly grind to a halt due to
                  swapping.

                  --
                  Fajar
                • Bill
                  Sorry I should have been more specific /swap.img is just a place holder value. You need to actually create it somewhere that is externally mounted. I do not
                  Message 8 of 10 , Mar 2 6:59 AM
                  • 0 Attachment
                    Sorry I should have been more specific /swap.img is just a place holder value. You need to actually create it somewhere that is externally mounted. I do not know if you are using SlugOS, or Optware with a custom firmware, Debian. With Debian /swap.img will work, because it won't be an in-memory device. If you / is a just a memory mount then it won't work, because linux is smart enough not to let you use a ramdisk or such as swapspace.

                    Bill

                    --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@...> wrote:
                    >
                    > I did use the mkswap command, just as noted in the how-to. I just tried again and got the same error message.
                    >
                    > Can you also address my other concern about the swap putting wear on the USB flash? Is it an issue to use the swapfile on the plug?
                    >
                    > So how does it work if the swapfile is larger than the actual memory?
                    >
                    > --- In nslu2-general@yahoogroups.com, "Bill" <yahoo@> wrote:
                    > >
                    > > You can make your swap file any size you want. (Within reason.) A small amount of non-swappable memory is used to map data in the swap file, so I would not recommend much more than 8 times the actual amount of memory you have on the device. Most people say 2 times the actual memory is a good size to use.
                    > >
                    > > Did you remember to use the mkswap command before activating your swap?
                    > >
                    > > Bill
                    > >
                    > >
                    > > --- In nslu2-general@yahoogroups.com, "adamant628" <adam.suhy@> wrote:
                    > > >
                    > > > I am pretty sure that I need to create some swap space, as one of the programs I am running is complaining on not having enough memory. I was following the instructions on http://www.nslu2-linux.org/wiki/HowTo/AddAdditionalSwapSpace to add a swap file. I am a linux newbie, but I have been learning. I successfully have programs running on my optware enabled pogoplug.
                    > > >
                    > > >
                    > > > Regarding the link above, the command: "dd if=/dev/zero of=/mnt/swapfile1 bs=1024 count=256k" is creating the swap file from the flash memory of the device? The how-to mentions that this will create a 256 MB swap file. I only have 125MB available on my device, so I assume that I shouldn't exceed that amount.
                    > > >
                    > > > I attempted to follow these instructions (made my swapfile 64MB), but when I used "swapon" I was given an error: "swapon failed: Invalid argument". This stumped me, so I don't know how to proceed at this point.
                    > > >
                    > > >
                    > > > My basic understanding of swap is that it is basically the same as virtual memory in windows. I have also read that it can burn out a USB flash drive fairly quickly due to the increased read/write cycles. Is it safe to do this on the pogoplug itself, or should I dedicate a portion of one of my externals for the task? I assume that if I use an external, I would skip the part about making the file, then direct mkswap to that partition's mount?
                    > > >
                    > > > Thanks for your help.
                    > > >
                    > >
                    >
                  • Bill
                    ... I agree. I used apache as a specific example, because it was one for awhile I had no choice to use. I had all the memory reducing options set at the
                    Message 9 of 10 , Mar 2 7:21 AM
                    • 0 Attachment
                      --- In nslu2-general@yahoogroups.com, "Fajar A. Nugraha" <fajar@...> wrote:
                      >
                      > On Fri, Mar 2, 2012 at 9:55 AM, Bill <yahoo@...> wrote:
                      > > Some programs such as apache took more memory than the NSLU2 had just to start-up.
                      >
                      > IMHO if you had programs like that, it's a sign of bad configuration.

                      I agree. I used apache as a specific example, because it was one for awhile I had no choice to use. I had all the memory reducing options set at the time, but during startup apache would still use more memory than an NSLU2 has available. Once started though, apache would drop it's memory usage to more reasonable.

                      Eventually, the bugs with lighttpd that prevented it from being usable for streaming videos were fixed and I switched to that. Although once running both lighttpd and apache use about the same amount of memory...

                      Bill
                    Your message has been successfully submitted and would be delivered to recipients shortly.