| 10:21 pm on Mar 29, 2003 (gmt 0)|
Got scootered for the first time as well. Will be a big ride as I can see for now.
Anyone can inform me how often there will be an update? Or do they add links continuesly, and if so, after how many days after the Scooter came by the links will be visible?
| 10:31 pm on Mar 29, 2003 (gmt 0)|
Altavista Launches New Look [webmasterworld.com].
Just started seeing Scooter/3.3_SF grabbing our pages that are refreshed.
Looks like there was a very recent update.
| 2:15 pm on Mar 30, 2003 (gmt 0)|
I just noticed that Altavista is spidering my pages like mad too. However, they're adding some text to the ends of the URLS and that is driving my rewrites crazy and hanging the server. Anyone else seeing this?
| 10:48 am on Apr 15, 2003 (gmt 0)|
I've got Scooter/3.3_SF coming every day pulling the same 3 pages. I didn't pay either.
Just today I noticed a 'referrer' or something from Scooter like this:
220.127.116.11 - - [15/Apr/2003:01:05:39 -0700] "GET /robots.txt HTTP/1.1" 200 2256 "htt*://www.some-site-that-has-nothing-to-do-with-mine/page.htm" "Scooter/3.3_SF"
Any ideas here?
| 5:25 pm on Apr 19, 2003 (gmt 0)|
Just noticed your question, mayor. I believe the answer is that AV fetches new copies of pages partially based on whether that page has been clicked on from AV's search results. So one way to encourage AV to crawl your site more often is to find your pages with a search, and then click to them.
| 5:21 am on Apr 21, 2003 (gmt 0)|
Thanks for the advice, GG. I'm ready for the Alta Vista stampede now, lol.
Kidding aside, though, making spidering frequency dependent on click rate does seem like a sensible way to keep from wasting spidering and processing bandwidth on pages that no one cares about.
| 2:47 am on Apr 22, 2003 (gmt 0)|
Check out this thread about AV and referrers:
| 5:25 pm on Apr 22, 2003 (gmt 0)|
hmmmm did somebody say acquisition?
oops overture did, me is retardo.