Page is a not externally linkable
- Google
-- Google News Archive
---- Google Asked To Hand Over Database Archives


the_nerd - 4:31 pm on Aug 12, 2004 (gmt 0)


ogletree,

There database is not what has valuable it is there algo and thee ability to dispense so many SERPS at once fast. Anybody can write a spider and collect the internet. I don't know what his problem is

But not every spider is allowed to spider as many pages a Google's. I'd probably block him because I see no commercial benefit to having somebody sucking in my pages.

nerd


Thread source:: http://www.webmasterworld.com/google_archive/25283.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com