Re: Tons of inetd errors -- any ideas why?
- It actually appears to be way weirder. I have a little script that does an emergency log dump that I can start and stop to see if there's any one IP pounding too hard on the server. I ran that yesterday for a few hours and found that rather than a traditional DOS, I'm being pounded by search engines.
Yahoo's doing a scan from multiple, sequential IP addresses, hitting my various sites (not just one!) at about 5 pages per second. Google's only using a few IPs, but also scanning about 2 pages per second. MSN is also spidering the pages. And there are about 10 other smaller-scale search engines doing the same thing.
I guess it could be the search engine equivalent of "that time of the month", but otherwise I can't imagine what's getting all the search engines to spider my pages at exactly the same time, with such a level of ferocity.
At first I thought that maybe someone with a top pagerank linked to me, but since they're scanning a few of my sites, not just one (although they all link together prominently), that doesn't seem to make as much sense.
Anyway, it's definitely impacting performance of the site (I don't think there's anything resembling polite spidering with these guys), but unless you can consider Yahoo and Google spidering as a DOS, it's legit access.
Weird and a little scary, though.
--- In firstname.lastname@example.org, Steve Hooker <steve@...> wrote:
> My guess would be a referer botnet that's got out of control.
> creecode wrote:
> > This would seem to be some kind of probing, attempted exploit or attack.
> > .
> > __,_._