Loading ...
Sorry, an error occurred while loading the content.

Re: Search Bot

Expand Messages
  • martin langhoff
    ... Doesn t Apache::Throttle, configured in apache to kick as a fixuphandler only for Googlebot, address the problem? I seem to recall a thread where people
    Message 1 of 7 , Jan 28, 2004
    • 0 Attachment
      >
      > Shawn wrote:
      >
      >>Hi, I have been trying to figure out a way to limit the massive amount
      >>of bandwidth that search bots (Googlebot/2.1) consume daily from my
      >>website. My problem is that I am running Apache::ASP and about 90% of

      Doesn't Apache::Throttle, configured in apache to kick as a fixuphandler
      only for Googlebot, address the problem?

      I seem to recall a thread where people were using it to throttle clients
      based on a "bad client" IP list that was updated on a cron script.

      cheers,





      martin

      --
      Reporting bugs: http://perl.apache.org/bugs/
      Mail list info: http://perl.apache.org/maillist/modperl.html
      List etiquette: http://perl.apache.org/maillist/email-etiquette.html
    Your message has been successfully submitted and would be delivered to recipients shortly.