homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

ATW! Make me smile and make me cry ...

 12:55 pm on Jan 21, 2003 (gmt 0)

Today, I saw ATW bot reaching my site. Was very excited and happy, but it only takes 20 files and sadly those files are NOT IMPORTANT at all pages. ATW bot come to my index page, it didn't crawl the link from the top to bottom. Instead it take the 1st link, get into it and get another 3 pages. It also get most of the links at the end of the root page instead of those important ones in the very beginning. Huh.





 3:33 pm on Jan 21, 2003 (gmt 0)

That does sound kinda screwy for a crawler in my opinion. Sounds kinda like a crap shoot...cover your bases and hope you make your point! :)


 6:10 pm on Jan 21, 2003 (gmt 0)

The bot still hasn't come back for the last 12 hours! huh ... come on baby, come on baby!


 8:22 pm on Jan 21, 2003 (gmt 0)

I bet it will come back soon. It has a_lot of sites to run through. Don't worry too much - make a few pages while you wait :)


 6:08 am on Jan 22, 2003 (gmt 0)

Hehe, just have a few link exchange while I wait ..

too bad wake up today buy Fast didn't pay a visit the whole night? :(

I think they are smart, they help me to crawl pages like - contact us, about us, links, terms of use, privacy policy ... so that I would need to pay to index my real contents. :) (just joking) Just wish they would index the complete site for the next update, instead of the next next update.


 6:55 am on Jan 23, 2003 (gmt 0)

It looks like ATW has just update their database. Most of the pages the crawler get 2 days ago appeared in th database! Wow, amazing on the speed of crawl -> database = 2 days. While google has a freshbot which is quite fast, but its real indexing happens only once a month and crawl -> database = 1 month!

Looks like I will need to wait for the next update before it come back to take the more valuable pages.

What is ATW update cycle? 2 weeks?


 9:15 am on Jan 23, 2003 (gmt 0)

I must say I've got the right SEO touch with ATW looking at the current results. If only Google were this easy. (rfgdxm grumbles about Google's excessive weighting of inbound link anchor text.)


 11:17 am on Jan 23, 2003 (gmt 0)

>What is ATW update cycle? 2 weeks?

Happens on a ongoing basis - smaller chunks at a time, around 25% I think. Sometimes we see a major update though.

Been a while since I really dug into it though. Anybody with more specific numbers?


 12:27 pm on Jan 23, 2003 (gmt 0)

Google's excessive weighting of inbound link anchor text

Hmm, this might be a good news or bad news. Since we know 'how it works' and after we do some 'keyword research', we have the ability to raise the ranking by using the correct anchor text. :)

But it is quite frustrated that google don't really care on the theme where the link is file. Google should check the title of the page with the link and compare it to its link to make them more relevant.

For ATW, sometimes I do see funny results appearing #2 and #3. It is better to be irrevalant so that people will still checking other sites below until they reach mine. haha. What's good on ATW is we don't need to wait so long to see the results appearing.


 7:11 pm on Jan 24, 2003 (gmt 0)

>It also get most of the links at the end of the root page instead of those important ones in the very beginning.

Same story with a site here. Had initially PFI'ed a couple of selling pages from the site. Now the main crawler has hit the site, took links from a secondary bottom navbar.
An interesting tidbit here is the free pages already appear in the db, but without the usual title and snippet.
Instead ATW gives the filename as title, and the linktext as snippet.


 3:11 am on Jan 25, 2003 (gmt 0)

ATW takes everything from the root directory /files.php and ignore those in subdir /subdir/files.php in my case


 10:15 pm on Jan 26, 2003 (gmt 0)

That's strange jamesyap. I have plenty of sites when pages in subdirectories rank just fine. I assume that you have links to those pages?

One reason could be the clustering/collapsing [webmasterworld.com] of pages from the same site that we've seen FAST do.


 7:37 am on Jan 27, 2003 (gmt 0)


My site is new, so ATW come step by step, the 1st update takes only the main page, 2 weeks later it follows all the links in the main page but in the main directory, I believe on the next update, it will reach all.

For the root index lost page bug, it happend to one of my site and friendly sole from ATW has help me to solve it.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved