Forum Moderators: open

Message Too Old, No Replies

Sohu crawl so fool?

         

Xuefer

4:11 pm on Nov 21, 2004 (gmt 0)

10+ Year Member



misunderstanding query_string.
hit same page by 2 bots as same time(with different cookie), with same ip
session cookie is treated as persist cookie

anyone watching it?

bill

12:13 am on Nov 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I'm not following you here Xuefer. Could you explain this in a bit more detail?

Xuefer

2:09 am on Nov 27, 2004 (gmt 0)

10+ Year Member



i monitored my pages
each page get hit by that bot twice, with same ip.
funny err?
wonder if anyone else doing same as me.

maybe it's the problem of my page that make the bot do so?
i'm not sure, but this bot even can't parse query_string correctly.

jdMorgan

4:18 am on Nov 27, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sohu also has an annoying habit of checking robots.txt *after* it attempts to fetch the index page.
No contact info in the user-agent string, either.

Jim