homepage Welcome to WebmasterWorld Guest from 54.227.231.229
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
disallowing a variable
KevinC




msg:3783672
 8:57 pm on Nov 10, 2008 (gmt 0)

disallow: /*?PageAction=EMAILFRIEND

just a quick question about the above? I want to disallow the "emailfriend" variable but not the pageaction variable as I require it for other pages that I want to stay indexed.

Is the above safe to just disallow the emailfriend link?

 

KevinC




msg:3783805
 12:43 am on Nov 11, 2008 (gmt 0)

I think I found my answer but I'm little nervous pulling the trigger.

does this look correct:

User-agent: *
Disallow: /admin/
Disallow: /affiliatewiz/
Disallow: /eproducts/
Disallow: /themes/

User-agent: googlebot
User-agent: slurp
User-agent: msnbot
Disallow: /*EMAILFRIEND*

goodroi




msg:3784071
 12:40 pm on Nov 11, 2008 (gmt 0)

that should work. be aware that wildcards are not official parts of robots.txt protocol. they are special allowances provided by the big search engines. be careful and keep and eye on it just to be safe.

KevinC




msg:3788301
 10:37 pm on Nov 17, 2008 (gmt 0)

thanks a lot, I have it in place and so far no issues.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved