Elliott C. Back: Internet & Technology

Today I had the pleasure of a random guy in Mexico recursively downloading as much of my site as he could, which sent my CPU load to 2.0, a level that Dreamhost would find acceptable but which I personally freak out about. The r-dns and IP of this guy are:

dsl-189-171-15-59.prod-infinitum.com.mx
189.171.15.59

He started at 04/Nov/2007:12:04:36 and ended (by iptables ban) at 04/Nov/2007:20:17:03. In those 8 hours and thirteen minutes, he made over 250,000 requests. That’s an extra 8.5 requests per second from a single IP, which is clearly unacceptable behavior:

[root@fc624389 ~]# cat access_log | grep 189.171.15.59 | wc -l
251923

If you don’t believe me, the next biggest offender over the last 24 hours made only 4,400 requests:

[root@fc624389 ~]# cat access_log | cut -d’ ‘ -f1 | sort -n | uniq -c | sort -nr | more
251923 189.171.15.59
4403 66.249.73.116
2012 76.88.78.239
1646 70.141.105.233

The user agent of this guy doesn’t tell *me* anything about him, but maybe one of you readers has an idea?

189.171.15.59 – - [04/Nov/2007:12:04:38 -0500] “GET /wp-content/themes/greenmarinee/images/links_bullet.gif HTTP/1.1″ 200 467 “http://celebrity-photos.elliottback.com/” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322)”

Another thing that bugs me is he requested each URL about 7 times. WTF? Do you really need to spider my site as fast as you can seven times?

[root@fc624389 ~]# cat access_log | grep 189.171.15.59 | cut -d’ ‘ -f11 | sort | uniq | wc -l
35414

I am either thinking of writing a very evil script to confuse non-google/msn/live/ask/yahoo bots by writing in an infinite number of invisible links into my websites, or installing some kind of mod_throttle into my apache. It looks like mod_limitipconn might help here, too.

Heh, my (x)html does not validate! At least, that’s what the w3c says after banning me for hitting refresh like twice in a single minute:

Service Temporarily Unavailable
The bwshare module will refuse your requests for the next 41 seconds.
You have made too many requests per second.

Apache/2.2.4 (Debian) mod_bwshare/0.2.0 Server at validator.w3.org Port 80

And just minutes later, it lets me in:

Congratulations

The document located at *blah* was checked and found to be valid XHTML 1.0 Transitional. This means that the resource in question identified itself as “XHTML 1.0 Transitional” and that we successfully performed a formal validation using an SGML or XML Parser (depending on the markup language used)

This is unacceptably lame; if the WWW Consortium is so strapped for money they cannot afford a couple boxes to run their validation service on, expect the organization to completely fall apart in the near future. Thanks for the HTML, it was nice.

Quote: “Go play counterstrike while I make you dinner dear.” OMG!