Page is a not externally linkable
- Search Engines
-- Ask - Teoma
---- Bot is killing my site, is it worth it?


moltar - 3:13 pm on Aug 8, 2003 (gmt 0)


Hello,

"Jeeves/Teoma" is spidering my site right now. I have many, many dynamic pages. That go like this - /country/state/city/ and I have almost every city in US and Canada. That makes ~ 20,000 pages. I have little content on those pages yet thou. And ask is getting *every single page*. I already reached 80% of my bandwidth usage just overnight.

Is it worth it? I didn't pay for inclusion, so I might not even get listed, right? And if I will get listed, will amount of pages reflect ranking?

And the same question for other SE, should I let them or it's not worth it?

Please respoint quick, because bot is still out there and I don't know what to do.

Thank you.

[edit]spelling and added a question[/edit]

[edited by: moltar at 3:28 pm (utc) on Aug. 8, 2003]


Thread source:: http://www.webmasterworld.com/ask_jeeves_teoma/316.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com