Need help about File Robots.txt

how do I know spiders visit to my site or not?

[viva]

Dear all

I have a little wonder about how spiders visit to my site.

Do we have any tricks to notice that spiders are already visited to the site? and how to check how many time that spiders are visited to my site?

Thank you in advance.

[Leosghost]

Ask your host to implement a stats programme that shows spider visits separate from other visits ( you can of course "mine" your "raw" stats for this but if you are new to doing this then a simple pre sorted prog is better ..
For example "awstats" is included in virtually any site running "Cpanel" as the management GUI..
To implement awstats can be a PITA if you have to switch it all in manually ..but in 9 from 10 "Cpanels" it is in the "Master panel options" as a simple "click for on"...

This will give you the robot names as they visit ..the number of "hits" per visit etc ..

[viva]

Thank you very much and I will try it.