the NEW Hotbot/Lycos alliance

Which spiders now? No more inktomi

[Friday]

HotBot used to be an easy one to get ranked highly in. Now I can't even figure out how to get in there!

Anyone know what's going on over there?

Are exodus spiders running the show?

Which blocks?

I have:
(64.75.0.0 - 64.75.63.255)

but only sure of:
64.75.36.*

[korkus2000]

Where did you here that hotbot was dropping inktomi?

Slurp is the spider for inktomi which has been feeding hotbot its results. Right now hotbot is using inktomi results.

[mbauser2]

Friday, is your problem about a previously-listed page dropping out of Hotbot, or are you trying to get a new site listed?

As korkus2000 said, Hotbot is still an Inktomi licensee, so you should be seeing "slurp" in your user_agent logs if Inktomi knows about your site. (Hotbot also get some results from Direct Hit, so hits from directhit.com's IP range would be a good sign, too.)

If you're trying to get a new site listed, you have three choices: paid submission, waiting for slurp to find you on its own, or find a free submission form on one of Inktomi's other partners. (MSN Bcentral and Hotbot UK still had free forms last time I looked.)

[misosoph]

Thank you for suggested sites. This free submit page I found:
www.hotbot.lycos.co.uk/submit.html
"Your site will be in the index within the next 30 days."

However, I could not find a free submit at www.bcentral.com , and I did not look at MSN ("... no going there by day, 'twas worse by night").

Re: inktomi "loops" (i.e. not indexing new/added pages on a site, but requesting the old pages again and again). Our host has written "The default with Inktomi is index,nofollow." www.searchengineworld.com/metatag/robots.htm

I therefore added:
<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">
to the Web page I submitted. I will wait to see if this has any effect on slurp because there are links on that page to new/added pages.

[misosoph]

Follow-up to getting Inktomi to break out of its loop (i.e. requesting the same pages over and over again, but ignoring newly added pages).

Submitting via hotbot.lycos.co.uk and the inclusion of the "index,follow" meta tag may work, because Inktomi is now reindexing my entire site, at a rate of 5-10 pages a day.

(However, also, my site's host changed my site's fixed IP number at the beginning of last month. So maybe that was a factor? About the possible side effects of having your site's static IP number changed, I am completely ignorant.)