homepage Welcome to WebmasterWorld Guest from 54.163.139.36
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

    
Lycos Insight Select
Editing chosen URL to get many pages spidered.
lavapies

10+ Year Member



 
Msg#: 643 posted 5:07 pm on Sep 21, 2002 (gmt 0)

Hi folks,

I just paid for 1 page of my site to be spidered by Lycos and get into FAST. I'd like the whole site spidered but can't afford the big fees for doing this.

Can I just edit my chosen URL for spidering once in a while and choose a different page on my site - and then eventually the whole lot will be spidered.

I guess this probably wouldn't work, but thought it may be worth asking anyway :-)

Thanks

 

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 643 posted 8:27 am on Nov 21, 2002 (gmt 0)

Nope, as far as I know, it is the single page only.

heini

WebmasterWorld Senior Member heini us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 643 posted 8:44 am on Nov 21, 2002 (gmt 0)

>edit my chosen URL for spidering once in a while and
choose a different page on my site

Well, you can do that. Free spidering of a site will anyhow not be influenced by that.
Free spidering can happen nonetheless. If you have the relevant links, like ODP etc, in my experience Fast will eventually come along to crawl the whole site.

Rumbas

WebmasterWorld Administrator 10+ Year Member



 
Msg#: 643 posted 10:04 am on Nov 21, 2002 (gmt 0)

Yes, you can change the URL but only within the same domain.

Of course the old url will drop out when the new one is indexed.

biggles

10+ Year Member



 
Msg#: 643 posted 12:32 am on Nov 26, 2002 (gmt 0)

I've always thought one of the key reason to consider paid inclusion is it allows indexing of dynamic URLs that would otherwise go unspidered. Certainly that's been my experience with AV & Inktomi.

Was puzzled as to why some pages from a ColdFusion site I'd submited to FAST via Lycos InSite Select weren't being indexed. After a bit of digging this is what I found:

Please do not submit web pages with these symbols in the URL: ampersand (& ), percent sign (%), equals sign (=), dollar sign ($) or question mark (?). Our spider does not recognize them.(my emphasis)

:( Not impressed at all and I'm annoyed this isn't more clearly highlighted. Certainly this is not made obvious when you're handing over your $ and submitting. And, surprise, dynamic URLs are accepted at this point. You only find out after the event.

Anyone know if this is the same if you submit to FAST via Position Tech?

Rumbas

WebmasterWorld Administrator 10+ Year Member



 
Msg#: 643 posted 8:08 am on Nov 26, 2002 (gmt 0)

Biggles, you should be able to submit almost any type of url through PT.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved