pardo
msg:21328 | 10:21 pm on Mar 29, 2003 (gmt 0) |
Got scootered for the first time as well. Will be a big ride as I can see for now. Anyone can inform me how often there will be an update? Or do they add links continuesly, and if so, after how many days after the Scooter came by the links will be visible?
|
coconutz
msg:21329 | 10:31 pm on Mar 29, 2003 (gmt 0) |
Altavista Launches New Look [webmasterworld.com]. Just started seeing Scooter/3.3_SF grabbing our pages that are refreshed. Looks like there was a very recent update.
|
gbaker123
msg:21330 | 2:15 pm on Mar 30, 2003 (gmt 0) |
I just noticed that Altavista is spidering my pages like mad too. However, they're adding some text to the ends of the URLS and that is driving my rewrites crazy and hanging the server. Anyone else seeing this? Thanks,
George
|
DavidT
msg:21331 | 10:48 am on Apr 15, 2003 (gmt 0) |
I've got Scooter/3.3_SF coming every day pulling the same 3 pages. I didn't pay either. Just today I noticed a 'referrer' or something from Scooter like this: 216.39.50.144 - - [15/Apr/2003:01:05:39 -0700] "GET /robots.txt HTTP/1.1" 200 2256 "htt*://www.some-site-that-has-nothing-to-do-with-mine/page.htm" "Scooter/3.3_SF" Any ideas here?
|
GoogleGuy
msg:21332 | 5:25 pm on Apr 19, 2003 (gmt 0) |
Just noticed your question, mayor. I believe the answer is that AV fetches new copies of pages partially based on whether that page has been clicked on from AV's search results. So one way to encourage AV to crawl your site more often is to find your pages with a search, and then click to them.
|
mayor
msg:21333 | 5:21 am on Apr 21, 2003 (gmt 0) |
Thanks for the advice, GG. I'm ready for the Alta Vista stampede now, lol. Kidding aside, though, making spidering frequency dependent on click rate does seem like a sensible way to keep from wasting spidering and processing bandwidth on pages that no one cares about.
|
fiestagirl
msg:21334 | 2:47 am on Apr 22, 2003 (gmt 0) |
DavidT
Check out this thread about AV and referrers:
[webmasterworld.com...]
|
cherryetc
msg:21335 | 5:25 pm on Apr 22, 2003 (gmt 0) |
hey whats this at the bottom of av... 'Effective April 23, 2003, AltaVista's Privacy Policy will change to reflect a pending acquisition.' hmmmm did somebody say acquisition? oops overture did, me is retardo.
|
|
