Loading ...
Sorry, an error occurred while loading the content.

Grouping BOT IPs

Expand Messages
  • FrancisChapman
    Grouping BOT IPs can improve visibility as to who is visiting your site. We ve had a large increase in site visits. After aggregating entries in the Sites
    Message 1 of 1 , Nov 7, 2005
    • 0 Attachment
      Grouping BOT IPs can improve visibility as to who is visiting your
      site. We've had a large increase in site visits. After aggregating
      entries in the Sites table I'm better able to see what proportion is
      due to increase in search engine crawler/spider visits.

      Post #3405 correctly pointed out it's best to use wildcards and not
      list individual IP addresses in a range, this example does that.

      I used SpiderTrack.org and IPLists.com to build the list. The
      following lines go into your Webalizer CONF file.

      GroupSite 66.249.* GoogleBot
      HideSite 66.249.*

      GroupSite 216.239.36.* GoogleBot_Translation
      HideSite 216.239.36.*

      GroupSite 216.239.38.* GoogleBot_Translation
      HideSite 216.239.38.*

      GroupSite 65.54.* MSNBot
      HideSite 65.54.*

      GroupSite 64.242.* LookSmart_Crawler
      HideSite 64.242.*

      GroupSite 66.147.154.* IBM_Almaden
      HideSite 66.147.154.*

      GroupSite 68.142.* Yahoo_Slurp
      HideSite 68.142.*
    Your message has been successfully submitted and would be delivered to recipients shortly.