Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Help to rightly create robots.txt


mcnika - 4:37 am on Sep 10, 2010 (gmt 0)


Hi.

I have dynamic urls like this.

mydomain.com/?pg=login
mydomain.com/?pg=reguser
mydomain.com/?pg=aboutus
mydomain.com/?pg=termsofuse

When the page is requested for ex. mydomainname.com/?pg=login index.php include login.php file.

some of the urls are converted to static url like
mydomain.com/aboutus.html
mydomain.com/termsofuse.html

I need to allow index mydomainname.com/aboutus.html, mydomainname.com/termsofuse.html
and disallow mydomainname.com/?pg=login, mydomainname.com/?pg=reguser, please help to manage my robots.txt file.

I have also mydomainname.com/posted.php?details=50 (details can have any number) which I converted to mydomainname.com/details/50.html
I need also to allow all this type of urls.


Thread source:: http://www.webmasterworld.com/robots_txt/4199823.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com