Welcome to WebmasterWorld Guest from 54.234.38.8

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
how to disallow shopping cart
rank is dropping since we set it up
2 Lorel 7:39 am Feb 12, 2006
Robots.txt help
Ok, maybe I did not understand how it works
15 followgreg 9:20 pm Feb 10, 2006
? and * in robots.txt
What stands a questionmark in the robots.txt for?
2 solandre 5:50 pm Feb 9, 2006
Meta tags and Robots
5 Buster42 4:48 pm Feb 9, 2006
Character Limit
Has anyone head of a character limit in robots.txt?
7 bumpaw 8:39 am Feb 9, 2006
Is this Robots.txt OK?
Please check this robot.txt file
9 mzconstruction 11:06 pm Feb 3, 2006
robots.txt and SSL
3 AlpacaAnnie 8:24 am Feb 3, 2006
Are different User-agent entries needed for versions of similar robot
3 Ceyx 2:29 am Feb 1, 2006
Robots list service?
Looking for a service provider or subscription service
2 xbase234 1:41 am Feb 1, 2006
How to disallow URLs with parameters?
"/page-name.php?" or "/page-name.php"
11 iProgram 11:34 pm Jan 31, 2006
This OK?
3 TonysDesigns 10:13 pm Jan 30, 2006
block robots from images
robot images
2 gb_dev 9:12 pm Jan 30, 2006
If my website dont have robots.txt
robots.txt
7 rusmart 8:50 pm Jan 30, 2006
Duplicate Content in Robots.txt file
2 palmpal 12:17 am Jan 30, 2006
To allow some robots only
9 GoldenHammer 3:14 pm Jan 28, 2006
robots.txt vs htaccess for control
if the robots.txt is not dependable why not just use htaccess?
6 ccubed99 3:39 am Jan 28, 2006
robots.txt and htaccess
4 agradation 11:17 pm Jan 26, 2006
Typical robots.txt?
Anybody got a same robots.txt?
1 1Lit 6:58 pm Jan 26, 2006
Disallow for sub-domain directory?
7 palmpal 4:18 pm Jan 26, 2006
Blocking old pages
1 Nutter 3:21 pm Jan 26, 2006
Wildcards special syntax for Google
publish Google unswer about special Google syntax in robots.txt plus examle
2 spiral 3:44 am Jan 26, 2006
Spiders finding you against all odds
3 Lyndsay 6:49 pm Jan 23, 2006
yahoo/slurp spidering
We don't want these pages indexed by yahoo
9 latimer 8:28 am Jan 23, 2006
Correct way to ban a directory
5 palmpal 3:18 am Jan 23, 2006
WWW vs. Non WWW
Site indexed twice need robot.txt help
3 carjocky 4:10 pm Jan 20, 2006