Loading ...
Sorry, an error occurred while loading the content.

Re: Ram-Usage of webalizer

Expand Messages
  • hansmeiserweb
    ... sry, not an option... Hans
    Message 1 of 7 , Mar 18, 2009
    • 0 Attachment
      > sure... transfer the logs to another machine and process them there ;)

      sry, not an option...

      Hans
    • waldo kitty
      ... well, i do hope you took it in the way it was intended... i was not joking and it is/was a valid solution that many folk do use... you might have to roll
      Message 2 of 7 , Mar 18, 2009
      • 0 Attachment
        hansmeiserweb wrote:
        >> sure... transfer the logs to another machine and process them there ;)
        >
        > sry, not an option...

        well, i do hope you took it in the way it was intended... i was not joking and
        it is/was a valid solution that many folk do use...

        you might have to roll your sleeves up and modify your webalizer to use disk
        files for its processing rather than it taking the (currently standard method
        used in many apps) current method of maintaining the lists in memory for fastest
        traversal and enumerating...

        barring that, would it be possible to add more memory to that machine for this
        situation? since the problem is memory consumption and swapping to disk, more
        physical RAM space would also be a good and likely the best solution...

        --
        NOTE: NEW EMAIL ADDRESS!!

        _\/
        (@@) Waldo Kitty, Waldo's Place USA
        __ooO_( )_Ooo_____________________ telnet://bbs.wpusa.dynip.com
        _|_____|_____|_____|_____|_____|_____ http://www.wpusa.dynip.com
        ____|_____|_____|_____|_____|_____|____ ftp://ftp.wpusa.dynip.com
        _|_Eat_SPAM_to_email_me!_YUM!__|_____ wkitty42 -at- windstream.net
      • hansmeiserweb
        ... this is not very practicable for us. we cannot do a sponsoring of another server for this costumer. also we can not do a daily manuell transfering, parsing
        Message 3 of 7 , Mar 19, 2009
        • 0 Attachment
          > well, i do hope you took it in the way it was intended... i was not joking and it is/was a valid solution that many folk do use...

          this is not very practicable for us. we cannot do a sponsoring of another server for this costumer. also we can not do a daily manuell transfering, parsing and 2nd transfering of files. this should be automated but this kind of action seems a bit far-fetched at this point.

          > you might have to roll your sleeves up and modify your webalizer to use disk disk files for its processing rather than it taking the
          > (currently standard method used in many apps) current method of
          > maintaining the lists in memory for fastest traversal and enumerating...

          thats the point. how? i did not find fitting options...

          > physical RAM space would also be a good and likely the best solution...

          may be, but i can not sell this to this costumer... not for this reason...

          hmm, need further suggestions. ;)

          Thanks,
          Hans
        • wstumper
          Since the server generating the large log files is running Apache, have you done anything to decrease log file size? Such as: SetEnvIfNoCase Request_URI
          Message 4 of 7 , Mar 20, 2009
          • 0 Attachment
            Since the server generating the large log files is running Apache, have you done anything to decrease log file size? Such as:

            SetEnvIfNoCase Request_URI "\.(gif)|(png)|(css)|(js)|(ico)|(jpg)|(pl)|(cgi)|(rdx)|(cgi)$" nolog
            CustomLog d:/logs/creation.log combined env=!nolog

            There may be many other types of files you don't need. Let Apache clean up the files rather than Webalizer. This might help a bit.

            For future reference you may want to consider a 'fork' of Webalizer by Stone Steps: http://www.stonesteps.ca/projects/webalizer/

            Performance:

            Stone Steps Webalizer running with a minimum set of group/hide filters, enabled GeoIP and disabled DNS resolver on a 3GHz Core 2 ™ Quad with 4GB of RAM can process 122161 log records per second. Adding dozens of filters, using robot processing and user agent name mangling will reduce processing speed to about 80000 records per second on the same system.

            Walt...

            --- In webalizer@yahoogroups.com, "hansmeiserweb" <hansmeiserweb@...> wrote:
            > thats the point. how? i did not find fitting options...
            >
            > > physical RAM space would also be a good and likely the best solution...
            >
            > may be, but i can not sell this to this costumer... not for this reason...
            >
            > hmm, need further suggestions. ;)
            >
            > Thanks,
            > Hans
          • Aaron Daniel Vega Villa
            why don t you make a container (if in solaris) or a VM inside your server so you will be able to run the webalizer on this sandbox and it will not consume
            Message 5 of 7 , Apr 22, 2009
            • 0 Attachment
              why don't you make a container (if in solaris) or a VM inside your server so you will be able to run the webalizer on this "sandbox" and it will not consume all your server memory, only the memory you allocated for the VM or container.

              --- El mié 18-mar-09, hansmeiserweb <hansmeiserweb@...> escribió:

              De:: hansmeiserweb <hansmeiserweb@...>
              Asunto: [webalizer] Ram-Usage of webalizer
              A: webalizer@yahoogroups.com
              Fecha: miércoles 18 de marzo de 2009, 5:24

              Hi,

              we have some problems running webalizer on special customer-server.
              This costumer has really huge logefiles. webalizer consumes 90% of Ram during creating stats. At this time also apache/mysql ist still running and it happens that server is running out of ram and swapfile is growing. So it happens that load climbs to "dizzy" regions and daemons are killed. In worst case server does a reboot.
              Is there a way to reduce max mem-usage of webalizer?

              Thanks,
              Hans




              ¡Obtén la mejor experiencia en la web!
              Descarga gratis el nuevo Internet Explorer 8.br> http://downloads.yahoo.com/ieak8/?l=mx
            Your message has been successfully submitted and would be delivered to recipients shortly.