Welcome to WebmasterWorld Guest from 54.197.159.173

Forum Moderators: open

Message Too Old, No Replies

How to get spidered?

     
6:55 am on Dec 30, 2001 (gmt 0)

Senior Member

joined:Oct 27, 2001
posts:10210
votes: 0


How long does it take for a new site to get spidered by Fast/Alltheweb, and what prompts the spider to pay a visit?

FWIW, my site went live on Oct. 11, had several listings in Yahoo by early November, and has had several thousand pages listed in Google since late November, but I've never seen a Fast or Alltheweb spider in my server logs. Is this normal?

10:11 am on Dec 30, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:Sept 25, 1999
posts:405
votes: 0


Had 2 sites launched oct. 18th, one got picked up by FAST and was indexed by the last update, the other didn't so I submitted it (december) to alltheweb.com and FAST was there to check, but no deepcrawl.

I'll have to wait for the next FAST update to see results for the site submittet.

I only submit the index page, like mydomain.com/

10:36 am on Dec 30, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 22, 2001
posts:2044
votes: 0


I've been waiting at least 6 weeks for a visit, with a listing in ODP; doesn't seem to be Fast to me.
Mind you, I'm also still waiting for Teoma and Wise Nut, so...
10:47 am on Dec 30, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Jan 31, 2001
posts:4404
votes: 0


>Is this normal?
Hard to tell at this point what is or is not normal with Fast.
Fast aims at agressively growing their index. Apparently they are encountering some problems lately. Update cycles have been delayed.

OTOH they are launching an PFI programme. The beta was said to be ended on 1 December. A European programme has been announced for several weeks. Neither thing happened.

europeforvisitors, Fast knows about your site, several links are in their index already. So I don't see any necessity to submit. That said it will not hurt to use the submission form at alltheweb.com

5:09 pm on Dec 30, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 16, 2001
posts:2059
votes: 0


>> OTOH they are launching an PFI programme. The beta was said to be ended on 1 December. A European programme has been announced for several weeks. Neither thing happened.

The second they are launching it you will now, i am V.I.P list for the PFI program :)
So you will get the URL before they launch it official :)

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members