Re: [webalizer] Re: Ram-Usage of webalizer
- hansmeiserweb wrote:
>> sure... transfer the logs to another machine and process them there ;)well, i do hope you took it in the way it was intended... i was not joking and
> sry, not an option...
it is/was a valid solution that many folk do use...
you might have to roll your sleeves up and modify your webalizer to use disk
files for its processing rather than it taking the (currently standard method
used in many apps) current method of maintaining the lists in memory for fastest
traversal and enumerating...
barring that, would it be possible to add more memory to that machine for this
situation? since the problem is memory consumption and swapping to disk, more
physical RAM space would also be a good and likely the best solution...
NOTE: NEW EMAIL ADDRESS!!
(@@) Waldo Kitty, Waldo's Place USA
__ooO_( )_Ooo_____________________ telnet://bbs.wpusa.dynip.com
_|_Eat_SPAM_to_email_me!_YUM!__|_____ wkitty42 -at- windstream.net
> well, i do hope you took it in the way it was intended... i was not joking and it is/was a valid solution that many folk do use...this is not very practicable for us. we cannot do a sponsoring of another server for this costumer. also we can not do a daily manuell transfering, parsing and 2nd transfering of files. this should be automated but this kind of action seems a bit far-fetched at this point.
> you might have to roll your sleeves up and modify your webalizer to use disk disk files for its processing rather than it taking thethats the point. how? i did not find fitting options...
> (currently standard method used in many apps) current method of
> maintaining the lists in memory for fastest traversal and enumerating...
> physical RAM space would also be a good and likely the best solution...may be, but i can not sell this to this costumer... not for this reason...
hmm, need further suggestions. ;)
- Since the server generating the large log files is running Apache, have you done anything to decrease log file size? Such as:
SetEnvIfNoCase Request_URI "\.(gif)|(png)|(css)|(js)|(ico)|(jpg)|(pl)|(cgi)|(rdx)|(cgi)$" nolog
CustomLog d:/logs/creation.log combined env=!nolog
There may be many other types of files you don't need. Let Apache clean up the files rather than Webalizer. This might help a bit.
For future reference you may want to consider a 'fork' of Webalizer by Stone Steps: http://www.stonesteps.ca/projects/webalizer/
Stone Steps Webalizer running with a minimum set of group/hide filters, enabled GeoIP and disabled DNS resolver on a 3GHz Core 2 Quad with 4GB of RAM can process 122161 log records per second. Adding dozens of filters, using robot processing and user agent name mangling will reduce processing speed to about 80000 records per second on the same system.
--- In email@example.com, "hansmeiserweb" <hansmeiserweb@...> wrote:
> thats the point. how? i did not find fitting options...
> > physical RAM space would also be a good and likely the best solution...
> may be, but i can not sell this to this costumer... not for this reason...
> hmm, need further suggestions. ;)
why don't you make a container (if in solaris) or a VM inside your server so you will be able to run the webalizer on this "sandbox" and it will not consume all your server memory, only the memory you allocated for the VM or container.
--- El mié 18-mar-09, hansmeiserweb <hansmeiserweb@...> escribió:
De:: hansmeiserweb <hansmeiserweb@...>
Asunto: [webalizer] Ram-Usage of webalizer
Fecha: miércoles 18 de marzo de 2009, 5:24
we have some problems running webalizer on special customer-server.
This costumer has really huge logefiles. webalizer consumes 90% of Ram during creating stats. At this time also apache/mysql ist still running and it happens that server is running out of ram and swapfile is growing. So it happens that load climbs to "dizzy" regions and daemons are killed. In worst case server does a reboot.
Is there a way to reduce max mem-usage of webalizer?
¡Obtén la mejor experiencia en la web!
Descarga gratis el nuevo Internet Explorer 8.br> http://downloads.yahoo.com/ieak8/?l=mx