- - Google
- -- Google News Archive
- ---- Google Asked To Hand Over Database Archives
the_nerd - 4:31 pm on Aug 12, 2004 (gmt 0)
|There database is not what has valuable it is there algo and thee ability to dispense so many SERPS at once fast. Anybody can write a spider and collect the internet. I don't know what his problem is |
But not every spider is allowed to spider as many pages a Google's. I'd probably block him because I see no commercial benefit to having somebody sucking in my pages.
Thread source: http://www.webmasterworld.com/google_archive/25283.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com