Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
I just paid for 1 page of my site to be spidered by Lycos and get into FAST. I'd like the whole site spidered but can't afford the big fees for doing this.
Can I just edit my chosen URL for spidering once in a while and choose a different page on my site - and then eventually the whole lot will be spidered.
I guess this probably wouldn't work, but thought it may be worth asking anyway :-)
Well, you can do that. Free spidering of a site will anyhow not be influenced by that.
Free spidering can happen nonetheless. If you have the relevant links, like ODP etc, in my experience Fast will eventually come along to crawl the whole site.
Was puzzled as to why some pages from a ColdFusion site I'd submited to FAST via Lycos InSite Select weren't being indexed. After a bit of digging this is what I found:
Please do not submit web pages with these symbols in the URL: ampersand (& ), percent sign (%), equals sign (=), dollar sign ($) or question mark (?). Our spider does not recognize them.(my emphasis)
:( Not impressed at all and I'm annoyed this isn't more clearly highlighted. Certainly this is not made obvious when you're handing over your $ and submitting. And, surprise, dynamic URLs are accepted at this point. You only find out after the event.
Anyone know if this is the same if you submit to FAST via Position Tech?