Loading ...
Sorry, an error occurred while loading the content.
 

Re: [webalizer] Re: Ram-Usage of webalizer

Expand Messages
  • waldo kitty
    ... well, i do hope you took it in the way it was intended... i was not joking and it is/was a valid solution that many folk do use... you might have to roll
    Message 1 of 7 , Mar 18, 2009
      hansmeiserweb wrote:
      >> sure... transfer the logs to another machine and process them there ;)
      >
      > sry, not an option...

      well, i do hope you took it in the way it was intended... i was not joking and
      it is/was a valid solution that many folk do use...

      you might have to roll your sleeves up and modify your webalizer to use disk
      files for its processing rather than it taking the (currently standard method
      used in many apps) current method of maintaining the lists in memory for fastest
      traversal and enumerating...

      barring that, would it be possible to add more memory to that machine for this
      situation? since the problem is memory consumption and swapping to disk, more
      physical RAM space would also be a good and likely the best solution...

      --
      NOTE: NEW EMAIL ADDRESS!!

      _\/
      (@@) Waldo Kitty, Waldo's Place USA
      __ooO_( )_Ooo_____________________ telnet://bbs.wpusa.dynip.com
      _|_____|_____|_____|_____|_____|_____ http://www.wpusa.dynip.com
      ____|_____|_____|_____|_____|_____|____ ftp://ftp.wpusa.dynip.com
      _|_Eat_SPAM_to_email_me!_YUM!__|_____ wkitty42 -at- windstream.net
    • hansmeiserweb
      ... this is not very practicable for us. we cannot do a sponsoring of another server for this costumer. also we can not do a daily manuell transfering, parsing
      Message 2 of 7 , Mar 19, 2009
        > well, i do hope you took it in the way it was intended... i was not joking and it is/was a valid solution that many folk do use...

        this is not very practicable for us. we cannot do a sponsoring of another server for this costumer. also we can not do a daily manuell transfering, parsing and 2nd transfering of files. this should be automated but this kind of action seems a bit far-fetched at this point.

        > you might have to roll your sleeves up and modify your webalizer to use disk disk files for its processing rather than it taking the
        > (currently standard method used in many apps) current method of
        > maintaining the lists in memory for fastest traversal and enumerating...

        thats the point. how? i did not find fitting options...

        > physical RAM space would also be a good and likely the best solution...

        may be, but i can not sell this to this costumer... not for this reason...

        hmm, need further suggestions. ;)

        Thanks,
        Hans
      • wstumper
        Since the server generating the large log files is running Apache, have you done anything to decrease log file size? Such as: SetEnvIfNoCase Request_URI
        Message 3 of 7 , Mar 20, 2009
          Since the server generating the large log files is running Apache, have you done anything to decrease log file size? Such as:

          SetEnvIfNoCase Request_URI "\.(gif)|(png)|(css)|(js)|(ico)|(jpg)|(pl)|(cgi)|(rdx)|(cgi)$" nolog
          CustomLog d:/logs/creation.log combined env=!nolog

          There may be many other types of files you don't need. Let Apache clean up the files rather than Webalizer. This might help a bit.

          For future reference you may want to consider a 'fork' of Webalizer by Stone Steps: http://www.stonesteps.ca/projects/webalizer/

          Performance:

          Stone Steps Webalizer running with a minimum set of group/hide filters, enabled GeoIP and disabled DNS resolver on a 3GHz Core 2 ™ Quad with 4GB of RAM can process 122161 log records per second. Adding dozens of filters, using robot processing and user agent name mangling will reduce processing speed to about 80000 records per second on the same system.

          Walt...

          --- In webalizer@yahoogroups.com, "hansmeiserweb" <hansmeiserweb@...> wrote:
          > thats the point. how? i did not find fitting options...
          >
          > > physical RAM space would also be a good and likely the best solution...
          >
          > may be, but i can not sell this to this costumer... not for this reason...
          >
          > hmm, need further suggestions. ;)
          >
          > Thanks,
          > Hans
        • Aaron Daniel Vega Villa
          why don t you make a container (if in solaris) or a VM inside your server so you will be able to run the webalizer on this sandbox and it will not consume
          Message 4 of 7 , Apr 22, 2009
            why don't you make a container (if in solaris) or a VM inside your server so you will be able to run the webalizer on this "sandbox" and it will not consume all your server memory, only the memory you allocated for the VM or container.

            --- El mié 18-mar-09, hansmeiserweb <hansmeiserweb@...> escribió:

            De:: hansmeiserweb <hansmeiserweb@...>
            Asunto: [webalizer] Ram-Usage of webalizer
            A: webalizer@yahoogroups.com
            Fecha: miércoles 18 de marzo de 2009, 5:24

            Hi,

            we have some problems running webalizer on special customer-server.
            This costumer has really huge logefiles. webalizer consumes 90% of Ram during creating stats. At this time also apache/mysql ist still running and it happens that server is running out of ram and swapfile is growing. So it happens that load climbs to "dizzy" regions and daemons are killed. In worst case server does a reboot.
            Is there a way to reduce max mem-usage of webalizer?

            Thanks,
            Hans




            ¡Obtén la mejor experiencia en la web!
            Descarga gratis el nuevo Internet Explorer 8.br> http://downloads.yahoo.com/ieak8/?l=mx
          Your message has been successfully submitted and would be delivered to recipients shortly.