THis might help:
Here [submitit.bcentral.com] you have a backdoor.
thank you for the interesting chart. The point is that I read somewhere that an inclusion in DMOZ would have granted an appereance in Inktomi as well.
The point is that I can't see how this could happen. Not even in Brett's chart this is shown in any way. Am I missing something?
AFAIK Inktomi spiders DMOZ, but doesn't use it "officialy".
Inktomi spiders the whole web, just like Google, so if there are links to your site (and that includes dmoz) they will eventually index you.
Wow, Thanks Tor!
How effective is this?
Not very. imho.
Depends on lots of things I do not understand. But to give you a picture of how difficult it can be:
I have 1700 links pointing to my site, (inc Yahoo and DMOZ) it has been up for nearly 1 year, but they only list my index page.
Google and alltheweb list 200+ pages so it is fully spiderable. From this, my best guess is that because I am on "shopping sites" they are waiting for me to pay. Not sure it is worth it, as I pay for Overture click throughs anyway and get little enough in the way of sales.
Same here George,
We have been deep crawled by every other SE but not
We are coming to the conclusion that this is because we
have used PFI around 5 months ago. Maybe they are waiting
for us to pay for further pages, so will not include any
We now have a new DMOZ and GOOGLE DIR listing in last few weeks,
so we are waiting to see if this makes any differance.
Call me a cynic, but no chance! It works for me in high value areas, (B2C) but not for selling socks :).
If you pay for inclution into Inktomi you are only paying for the pages you pay to be included. Any other pages on your site that get spidered are a bonus but don't count on it.
You are probably better off never paying them because if you pay for one pay for one page they'll probably wait for you to pay for others...
Inktomi spidered my site many times over a period of about 6 months, but none of my pages showed up in the index.
At that point, I bought 2 pay for inclusion "slots" - just for testing some pages.
About a month or two later most of the pages from my site appeared in the Inktomi index. And my site continues to be spidered by Inktomi and my new pages continue to appear in the index.
I'm not saying that the pay-for-inclusion caused my other pages to appear. It was most likely just coincidental timing.
Just want people to know that - at least in my case - pay for inclusion does not mean your other pages won't be spidered.
That is good news but you were spidered before
PFI, where as we had PFI from the begining and have never been spidered.
BTW did you have a DMOZ and GOOGLE DIR link before
you got spidered?
Perhaps those who pay-for-inclusion get on the "sucker list". Why spider someone for free if you know they are willing to pay. Those who never pay probably have a better chance for free spidering because if they didn't do freebees the search engine would be not be comprehensive enough to be relevant.
I was spidered by Inktomi long before I had the DMOZ listing - but without any pages being indexed.
It wasn't until a couple of months after I had the DMOZ listing that any pages from my site showed up in the Inktomi index. Don't know if the DMOZ listing made a difference. (I got accepted into DMOZ and did the Inktomi PFI at about the same time.)
To correct my previous post:
I just checked my notes and I see that, at first, only a few "free" pages showed up in Inktomi (about 2 months after PFI). Then a few more pages, etc. It took about four months before my site was getting a really deep spidering.
I have a sinking feeling that it is not the case with me. I was in DMOZ last May. Nearly 12 months now! How time flys!
Still nothing much.