excite, lycos, N.Light

[caine]

Looking at my logs, i could see that excite had two crawlers come and visit - storm and chokey, which made one request each, lycos sent about 20 crawlers, and must of nearly went round all my links, no indications of Northern Lights crawler, which i understood to be Gulliver.

Will excite come back to the site and do a proper index, as it is holding references to pages that existed about 3 months ago and probably submitted 3-5 years ago.

I am hopefully assuming since lycos sent all these various crawlers that they are intending to add the site and its pages to its database, though no evidence of this with either Lycos - Fast or Hotbot.

How long does it take Northern Light's Gulliver or is there any other names of its spiders - i submitted 8 weeks ago!

[jeremy goodrich]

I've seen them take as long as 2 months or longer. Gulliver is the one you are after, I think that if you are well linked in places, you should be able to get hit by him within a reasonable amount of time.

The nice thing about N.Light is that once you are in their db, they seem to update your info fairly regularly. Hope this helps.

Lycos crawlers to the international work, and Fast does the work for their domestic engine. You can submit to them at htp://www.alltheweb.com/

As for excite, rumor is they might be going under, or PFP (pay for performance). I don't have any info other than that which is in the other recent threads here. Happy reading, and hopefully they will update their db soon.

[caine]

Jeremy

Cheers for the advice. One other question
Should i submit all of my pages to the engines?

[jimmykav]

I get regular visits from gulliver every 2 -3 days and it fetches a page or three each time.....
it is one of themost reliable spiders out there from whqat i can see

[jeremy goodrich]

I would recommend submitting index or root level pages only to google, they will crawl the rest. Other than that, stick to submitting no more than about 5 per day.

If your site is well linked from DMOZ, Yahoo, Looksmart, etc. it may not be necessary at all to submit them.