homepage Welcome to WebmasterWorld Guest from 54.198.224.121
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Disallow indexing on dynamic files
Disallow indexing on dynamic files
premzero




msg:1529004
 1:29 am on Dec 10, 2003 (gmt 0)

On my website, I have a few dynamic pages generated using one file. Say x.cfm.
So the resulting dynamic pages are:
/x.cfm?a=987&b=787
/x.cfm?a=987
etc...

You get the idea.

How do I disallow the search engines from indexing any link with x.cfm? I'd appreciate the full correct code since I don't know much about robots.txt formatting.

Thanks

 

pageoneresults




msg:1529005
 1:37 am on Dec 10, 2003 (gmt 0)

Hello premzero, Welcome to WebmasterWorld!

User-agent: *
Disallow: /x.cfm?

That should do it.

Validate your robots.txt file here...

robots.txt Validator [searchengineworld.com]

pageoneresults




msg:1529006
 1:48 am on Dec 10, 2003 (gmt 0)

I'd appreciate the full correct code since I don't know much about robots.txt formatting.

Open up Note Pad. Cut and paste the code above into NP. Do a save as robots.txt. Upload to the root level of your web. It must reside in the root level, the URI should look like this...

http*//www.example.com/robots.txt

premzero




msg:1529007
 4:59 am on Dec 16, 2003 (gmt 0)

Sorry I've been away.

Thanks for the reply. I'm implementing it right away!

Cheers!

tenaka




msg:1529008
 2:51 pm on Dec 25, 2003 (gmt 0)

what permissions to give the robots.txt 755?

bill




msg:1529009
 2:58 am on Dec 26, 2003 (gmt 0)

what permissions to give the robots.txt 755?
644 should suffice
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved