Strange results from alltheweb

Google's fine, but Alltheweb...

[transistor]

I got spidered by Google in early september and in october wham! I was there and in very good position (thank you all at WebMasterWorld) for my keywords (artists, albums and songs names).

At the same time, I was visited by Fast-Alltheweb and finally my site shows up, but with a strange difference.
All the pages in my site appear without the Title and Description, except for the main index page and other 4 or 5 static pages.
This hurts my position, because my pages show up veeeeery late in the results.

To my knowledge, there's nothing fancy about the dynamic URLs (Google's got them just fine) that might impede a spider from getting the information of each page.
I've checked for validation from several tools and they all pass.
I apologize if I can't make myself clear.

If optimizing for Google, is similar to Alltheweb, then, what is going on?
Is it the way Alltheweb works with free indexing?
Or am I doing something wrong?

[McMohan]

Hi transistor,

Yup.. all that u need to do is go for Lycos InSite Select, paying $18 as Annual fee and $12 per page. Once u do that after 2 days u'll be on the first page on Lycos and ATW. Speakin out of experience.

[Shakil]

McMohan could you care to enlighten us with some more info, as I am sure life is not that easy.

how many other listings were their for the keyword which you have experimented with?

Shak

[heini]

Transistor, I would suggest it's a matter of time.
I would wait for another couple of weeks and see if Fast comes around to complete indexing all the pages with proper title and description.
You might consider PFI, which would speed up the process.
Generally Fast is quite good at indexing dynamic pages.

[jdMorgan]

transistor,

...without the Title and Description

This sounds like Fast found the links, but did not spider the pages at the time the latest index was generated. I'll go with heini on this one: Wait for the next update, unless you really want to pay and take your chances with insite...

Jim

[steveb]

InSite select is currently experiencing severe problems which cause sites to dramamtically lose ranking after they pay. No one should pay for that "service" InSite Select until this problem is fixed.

[heini]

To put this into perspective: a few people have had this problem. Fast says they are trying to get this worked out on an individual basis, as well as fixing the source of the problem.
A Fast rep has signed on to WebmasterWorld offering personal assistance to the members here:
[webmasterworld.com...]

[steveb]

"To put this into perspective: a few people have had this problem."

The fact that a FAST representative had to come to this board at all about this issue should tell any objective observer that it is a serious, widespread problem that they need to deal with and apparently are dealing with.

[edited by: heini at 11:56 pm (utc) on Oct. 20, 2002]

[newsparrot]

At a guess I would have initially said our site had been “struck off” as it was on #1 for lots of keywords as in google, until the last update, then it was gone!

Some pages are still showing and the links show up for links:. Would you think it was just a server clitch, or can we have done a baddie thing, having said that google came though like a train** last time and we did well? But does atm have a different set of rules?

[transistor]

Thank you all for your replies.
I think I'll first wait, like jdMorgan suggests.
I've read the posts about the ranking problems after paying, not that it would hurt *my* ranks! :D
Anyway, I'll keep my eyes open for the Alltheweb spider and let you know if this situation is corrected in the following updates.
Thanks again!

[Rumbas]

>But does atw have a different set of rules?

Yes, very much. FAST's raking criteria is somewhat different from Google's. For some pointers, check these threads:

How Alltheweb rank web pages? [webmasterworld.com]
Small faq/information about FAST. [webmasterworld.com]

BTW, welcome to WebmasterWorld newsparrot :)

[quiet_man]

transistor - my experience is that FAST is actually better than Google at spidering dynamic pages. However, it takes much, much longer to get properly listed in FAST than Google. I'm talking months here, so if you were only spidered by FAST in September then I would expect it to take another month or more for a full listing. (Can't explain your 'partial' listing details though).

[fctoma]

I just did a few searches for my domain on Google and FAST. I pay for FAST paid URL inclusion on some of my URLs.

Results:

Google had 45 pages.
FAST had 8 pages.

This seems consistent with our web logs. Not sure if the FAST URL inclusion just includes the paid ones or not. We had a bigger presence in FASTs index months ago when we didn't pay for their URL inclusion.

Best,
Frank

[heini]

>FAST URL inclusion just includes the paid ones or not

Definitely. PFI is pay per page.
When submitting you need to give the exact urls of the pages you want indexed. If you pay for one url and give them your index page than that's what gets spidered and refereshed each 24-48 hours for one year.
That's the deal.