Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

What is a web crawler?


5:00 am on Feb 23, 2010 (gmt 0)

New User

5+ Year Member

joined:Feb 23, 2010
votes: 0

System: The following 2 messages were cut out of thread at: http://www.webmasterworld.com/search_engine_spiders/4083253.htm [webmasterworld.com] by incredibill - 12:43 am on Feb 23, 2010 (PST -8)

What is a web crawler or a robot? Is it some software? Sorry for my complete ignorance but , I have never got a real answer to this ever.
5:34 am on Feb 23, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2038
votes: 1

You'll find the answers right under your fingertips. For starters, read the first few paragraphs of:

Wikipedia: Web crawler [en.wikipedia.org]

See also:

Wikipedia: Internet bot [en.wikipedia.org]

Want more info? Pick your choice of major search engine and go for it!