Welcome to WebmasterWorld Guest from 50.16.126.107

Forum Moderators: goodroi

Message Too Old, No Replies

Is this format ok please ?

     
9:27 pm on Jan 22, 2011 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 26, 2004
posts: 377
votes: 0


Been working on this for a while and feel I'm almost there. If you didn't want any images on your site to be indexed by the search engines and you only wanted Google, Yahoo, MSN and Ask jeeves to visit your website, would the below be ok please ?

User-agent: *
Disallow: /images/
Disallow: /a_images/
Disallow: /cgi-bin/

User-agent: Googlebot-Image
Disallow: /

User-agent: Yahoo-MMCrawler
Disallow: /

User-agent: psbot
Disallow: /



User-agent: *
Disallow: /

User-agent: Googlebot
Disallow:

User-agent: Slurp
Disallow:

User-agent: msnbot
Disallow:

User-agent: Teoma
Disallow:

Any help appreciated.

Dexie
11:49 am on Jan 25, 2011 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3171
votes: 153


it looks like you have two robots.txt files there. you have "User-agent: *" listed twice. you should only list it once.
11:09 am on Feb 25, 2011 (gmt 0)

New User

5+ Year Member

joined:Jan 20, 2011
posts:14
votes: 0


Yes i agree with goodroi.You are using two times robots in this format.Please use this for once.I think the rset is fine and ok.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members