Control Robots with Meta or robots.txt - Sitemaps, Meta Data, and robots.txt forum at WebmasterWorld - WebmasterWorld

Forum Moderators: goodroi

Message Too Old, No Replies

Control Robots with Meta or robots.txt

driller41

11:47 am on Jan 15, 2008 (gmt 0)

10+ Year Member

Hi, is it best to have a robots.txt file or include the statement in the metatag of the header?

Lord Majestic

11:53 am on Jan 15, 2008 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

robots.txt is the best way to control good bots.

phranque

12:21 pm on Jan 15, 2008 (gmt 0)

WebmasterWorld Administrator

10+ Year Member

Top Contributors Of The Month

you might find this thread from the robots.txt forum library [webmasterworld.com] to be useful start on the subject.