Forum Moderators: open
www.domain.com/index.html
www.domain.com/products/index.html
www.domain.com/products/typeofproduct/index.html
www.domain.com/products/typeofproduct/specificproduct/index.html
The site is themed so there is cross-linking across levels and linking down the tree to pages below that are related. I thought this would be great for spiders like FAST but there's a big problem right now. FAST is coming through and spidering my pages that have links like:
<a href="/products/typeofproduct/specificproduct/">Specific Product</a>
But instead of following the link he requests:
/products/specificproduct/
Why GOD??? Is anyone else seeing this. He's relentless in doing this even though none of the links are relative in terms of using ../blah or ./blah ... all are served up relative to the root directory with their full path (but not [domain.com...] I guess my beautiful pages won't be listed in FASTs index on the next update ... but what can I do to avoid this happening in the future?
Is the type of link you're refering to from your site in the profile? If so, I am a little confused as to what the problem is. Can you exlain a little further?