Forum Moderators: DixonJones

Message Too Old, No Replies

Where is the WebTrendsLive Bot Info?

         

Fruit and Veg

9:42 pm on Sep 9, 2003 (gmt 0)

10+ Year Member



Where can I find if search engine bots have crawled on a WebTrendsLive account?

I feel like Stevie Wonder looking for The Invisible Man in a dark room.

Thanks.

Fruit and Veg

2:13 pm on Sep 12, 2003 (gmt 0)

10+ Year Member



Here's the reply from WebTrends...

Unfortunately Webtrends Live does not track spider generated traffic. This is because spiders do not execute javascript code which is essesntial to our Webtrends Live tracking.

Bummer.

DaveN

2:15 pm on Sep 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Fruit and Veg,

thanks for the feedback.

DaveN

Fruit and Veg

2:31 pm on Sep 12, 2003 (gmt 0)

10+ Year Member



No worries. Any suggestions for spying spiders without having to download log files and running them offline would be appreciated.

I'm not going to get obssessed about checking for spiders every hour, but a monthly check to see who's been and who hasn't would be nice.

Fruit and Veg

5:55 pm on Sep 12, 2003 (gmt 0)

10+ Year Member



Just spied this great thread - [webmasterworld.com...]

Q7 states;

I haven't got access to my log file, how do I see if Googlebot spiders my pages.

Use a server side include (eg. Apache XSSI, PHP or ASP) to embed or call a script that checks for:
"Googlebot/2.1 (+http://www.googlebot.com/bot.html)" as the USER_AGENT or "crawl*.googlebot.com" or "crawler*.googlebot.com" as the HOST. You cannot use an image or Javascript based tracker because Googlebot won't trigger it.

Does anyone have the relevant SSInclude for an .asp page that I can just copy and paste onto a page? Also, could this bit of code be extended to include other bot info?