AllTheWeb

Why is only my Front Page crawled by AllTheWeb?

[Miop]

Hi,
I have an ecommerce site that has almost 1000 links, including 4 hundred and something on AllTheWeb, and my key pages are in the top 20 on Google. I used to have lots of pages listed in AllTheWeb, but now it's just the index page that is updated regularly and the others are gradually dropping out - they don't appear to have been indexed for over 6 months now. I don't pay to get listed in AllTheWEb - I was already listed from when it was free. I have been submitting my missing pages for some months, but they are not being indexed.
My site is totally static at present and always has been.
This has created a serious problem for me, as I have been fine tuning the seo of my site (I am all self-taught and it's taken a while!). My site used to appear for many keywords applicable to my site because I had crammed them all onto the first page. Now I am doing it properly, I am dropping down the indices where I am not spidered regularly because those keywords are not appearing. I can't afford paid inclusion to AllTheWeb - I am a one man small business and £140 pages would cost me megabucks!
Please could anyone tell me why AllTheWeb doesn't index my site?

I find it all a bit discouraging that when you bother to tune your seo to play by all the rules, you end up not being found. On Google this is not a problem and I have managed to get to number 1 for a few phrases because they spider me twice a week.
Thanks for your advice.

[Shawn607]

The only idea that I could come up with is that AllTheWeb is cutting down on some of your pages due to the fact that either you don't have them listed in the root directory or they are, for some reason, cutting down on websites listed to only one page per domain? I also read that you were able to get updated on google.com every two weeks and you also stated that you are a small business with a small amount of money to put towards advertising expenses. I would like to know how you went about getting updated twice a week and what the price is if any is applicable.

[heini]

>others are gradually dropping out - they don't appear to have been indexed for over 6 months now

What do you mean by dropping out - they are not in the index anymore, or they don't rank as well as before?
Did you check your logfiles for Fast Crawlers?

[Miop]

Thanks for the replies.

The pages are just one by one not being listed anymore on ATW. If I check every few weeks another couple are gone.
The ones that are there haven't been updated for about 3- 6 months, except the Index page. The FAST crawler only spiders the index page of my site now (I can see this from the logs).

As for Googlebot, I don't pay anything for it to come regularly - it just does. It doesn't always spider all the pages when it comes - it does this about once a month or so, but it usually spiders the main pages when it comes.

It never used to come that often.
I do have a Google Search box on my site, and I do use adwords, but don't they claim that it doesn't make any difference?
I have no idea why it comes that often, but I am glad it does as it makes seo much easier!