Welcome to WebmasterWorld Guest from 54.162.155.183

Forum Moderators: open

Message Too Old, No Replies

Why did AltaVista spider 160,000 pages

Recently joined the "light web"

     
9:07 pm on Jul 11, 2003 (gmt 0)

10+ Year Member




Why did AltaVista spider 160,000 pages if it wasn't going to do anything with them?

Just grumping mostly, but its been two months now since I changed the site to use spiderable links, and one week after I did that, AV pulled all 160,000 pages from our site ( I wrote a perl script to check, and the links were unique).

We still only have the old 30 pages in altavista though, with the old style links.

Grump. Grump. Grump. Yet the have an up to date copy of the home page.

10:42 pm on Jul 11, 2003 (gmt 0)

WebmasterWorld Senior Member jeremy_goodrich is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Perhaps one day, AV will return to former glory, however, every time somebody at WebmasterWorld asserts, "AV will make a come back" it seems that hope is ebbed further away...

At one point, I even took the time to ban their spiders from my sites (bandwidth drain) but these days, I hardly notice or care. ;)

10:50 pm on Jul 11, 2003 (gmt 0)

10+ Year Member



So basically, their spiders a running around the web slurping up sites, but not actually doing anything with the data?

Huh. So how come AV slurps 160,000 pages, but "All The Web" did less then Google?

Ah well. Can't wait until the grub.org guys get a Mac OS X version of their client running, so I can throw 160,000 pages at them. :-)

11:09 am on Jul 12, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Typically, it will take AV something between two and four months to actually index the pages they spidered. At least that was the rate last time I checked. So you might be just a little too impatient in this case.
11:35 am on Jul 12, 2003 (gmt 0)

WebmasterWorld Senior Member heini is a WebmasterWorld Top Contributor of All Time 10+ Year Member



AV has in the top serps a very fresh index, likely the freshest of all engines. So it looks like their policy is to keep top listings very fresh, and negelecting freshness for secondary results a bit.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month