Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Can IMDB do this?

charge per unwanted request

9:46 pm on Apr 19, 2004 (gmt 0)

New User

10+ Year Member

joined:May 14, 2002
votes: 1

I was looking at imdb.com's robots.txt, and noticed at the bottom:

# Robots that cause a denial of service will be charged $0.01 per unwanted
# request to this site. Follow the guidelines + robots.txt convention and
# we can all live happily together.

Can they legally charge someone $0.01 per unwanted request, or is this just more of a scare tactic?

9:51 pm on Apr 19, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 28, 2003
votes: 0

They can do anything they want. But collecting the money might be a completely different problem.
10:19 pm on Apr 19, 2004 (gmt 0)


WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 2, 2000
votes: 0

Jrap, that's pretty funny, I haven't seen it before. I guess if they sue someone, they can throw this provision in along with the kitchen sink when they tally up damages.

It seems like robots.txt has occasionally been shown as valid by courts, at least as far as determining whether someone had the right to spider another site. Since robots can't read the amount they are supposed to pay, though, I think the site owner would have a lot of difficulty proving that an enforceable contract exists.

"Unwanted" is kind of nebulous, too, as is the exact meaning of "denial of service". On the other hand, if they were getting persistent abuse by a robot and they served written notice to the abuser, it might be a more interesting day in court.