I use SSI for a virtual include on my navigation, is the spider going to pick this up and follow the links? I do have a few links in the contest, but the main ones are on the navigation bar wich is a virtual include. Thank you
deejay
6:38 am on Sep 27, 2003 (gmt 0)
Short answer: yes.
Long answer: yes. the script is 'server side', so the include becomes part of the file before the file is served up to the spider/viewer.
ganderla
6:44 am on Sep 27, 2003 (gmt 0)
Why thank you, that reason I ask is that i get plenty of daily spiders but, from what i can tell from my logs, there are just hitting the home page. Say for instance, the altavista bot comes for one visit there are only one or two hits. I have no robots.txt file
deejay
6:48 am on Sep 27, 2003 (gmt 0)
The one page spider visit is very common - I wouldn't worry about it.
At least it means the spiders know about your site. They'll be back for more sooner or later.