Welcome to WebmasterWorld Guest from 220.127.116.11
Forum Moderators: open
I have 1700 links pointing to my site, (inc Yahoo and DMOZ) it has been up for nearly 1 year, but they only list my index page.
Google and alltheweb list 200+ pages so it is fully spiderable. From this, my best guess is that because I am on "shopping sites" they are waiting for me to pay. Not sure it is worth it, as I pay for Overture click throughs anyway and get little enough in the way of sales.
We have been deep crawled by every other SE but not
We are coming to the conclusion that this is because we
have used PFI around 5 months ago. Maybe they are waiting
for us to pay for further pages, so will not include any
We now have a new DMOZ and GOOGLE DIR listing in last few weeks,
so we are waiting to see if this makes any differance.
You are probably better off never paying them because if you pay for one pay for one page they'll probably wait for you to pay for others...
At that point, I bought 2 pay for inclusion "slots" - just for testing some pages.
About a month or two later most of the pages from my site appeared in the Inktomi index. And my site continues to be spidered by Inktomi and my new pages continue to appear in the index.
I'm not saying that the pay-for-inclusion caused my other pages to appear. It was most likely just coincidental timing.
Just want people to know that - at least in my case - pay for inclusion does not mean your other pages won't be spidered.
I was spidered by Inktomi long before I had the DMOZ listing - but without any pages being indexed.
It wasn't until a couple of months after I had the DMOZ listing that any pages from my site showed up in the Inktomi index. Don't know if the DMOZ listing made a difference. (I got accepted into DMOZ and did the Inktomi PFI at about the same time.)
To correct my previous post:
I just checked my notes and I see that, at first, only a few "free" pages showed up in Inktomi (about 2 months after PFI). Then a few more pages, etc. It took about four months before my site was getting a really deep spidering.