homepage Welcome to WebmasterWorld Guest from 54.227.20.250
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Disallow indexing on dynamic files
Disallow indexing on dynamic files
premzero

10+ Year Member



 
Msg#: 215 posted 1:29 am on Dec 10, 2003 (gmt 0)

On my website, I have a few dynamic pages generated using one file. Say x.cfm.
So the resulting dynamic pages are:
/x.cfm?a=987&b=787
/x.cfm?a=987
etc...

You get the idea.

How do I disallow the search engines from indexing any link with x.cfm? I'd appreciate the full correct code since I don't know much about robots.txt formatting.

Thanks

 

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 1:37 am on Dec 10, 2003 (gmt 0)

Hello premzero, Welcome to WebmasterWorld!

User-agent: *
Disallow: /x.cfm?

That should do it.

Validate your robots.txt file here...

robots.txt Validator [searchengineworld.com]

pageoneresults

WebmasterWorld Senior Member pageoneresults us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 1:48 am on Dec 10, 2003 (gmt 0)

I'd appreciate the full correct code since I don't know much about robots.txt formatting.

Open up Note Pad. Cut and paste the code above into NP. Do a save as robots.txt. Upload to the root level of your web. It must reside in the root level, the URI should look like this...

http*//www.example.com/robots.txt

premzero

10+ Year Member



 
Msg#: 215 posted 4:59 am on Dec 16, 2003 (gmt 0)

Sorry I've been away.

Thanks for the reply. I'm implementing it right away!

Cheers!

tenaka

10+ Year Member



 
Msg#: 215 posted 2:51 pm on Dec 25, 2003 (gmt 0)

what permissions to give the robots.txt 755?

bill

WebmasterWorld Administrator bill us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 215 posted 2:58 am on Dec 26, 2003 (gmt 0)

what permissions to give the robots.txt 755?
644 should suffice
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved