Welcome to WebmasterWorld Guest from 54.145.118.24

Forum Moderators: goodroi

Message Too Old, No Replies

Need help about File Robots.txt

how do I know spiders visit to my site or not?

     
3:13 am on Nov 4, 2004 (gmt 0)

New User

10+ Year Member

joined:Feb 5, 2004
posts:15
votes: 0


Dear all

I have a little wonder about how spiders visit to my site.

Do we have any tricks to notice that spiders are already visited to the site? and how to check how many time that spiders are visited to my site?

Thank you in advance.

7:29 am on Nov 4, 2004 (gmt 0)

Senior Member from FR 

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Feb 15, 2004
posts:6717
votes: 230


Ask your host to implement a stats programme that shows spider visits separate from other visits ( you can of course "mine" your "raw" stats for this but if you are new to doing this then a simple pre sorted prog is better ..
For example "awstats" is included in virtually any site running "Cpanel" as the management GUI..
To implement awstats can be a PITA if you have to switch it all in manually ..but in 9 from 10 "Cpanels" it is in the "Master panel options" as a simple "click for on"...

This will give you the robot names as they visit ..the number of "hits" per visit etc ..

10:15 am on Nov 4, 2004 (gmt 0)

New User

10+ Year Member

joined:Feb 5, 2004
posts:15
votes: 0


Thank you very much and I will try it.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members