Welcome to WebmasterWorld Guest from 107.20.20.39

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Why doesn't this validate?
Help please
4 Vimes 11:04 am Feb 14, 2006
TripAdvisorBot?
1 jexx 9:02 pm Feb 13, 2006
how to disallow shopping cart
rank is dropping since we set it up
2 Lorel 7:39 am Feb 12, 2006
Robots.txt help
Ok, maybe I did not understand how it works
15 followgreg 9:20 pm Feb 10, 2006
? and * in robots.txt
What stands a questionmark in the robots.txt for?
2 solandre 5:50 pm Feb 9, 2006
Meta tags and Robots
5 Buster42 4:48 pm Feb 9, 2006
Character Limit
Has anyone head of a character limit in robots.txt?
7 bumpaw 8:39 am Feb 9, 2006
Is this Robots.txt OK?
Please check this robot.txt file
9 mzconstruction 11:06 pm Feb 3, 2006
robots.txt and SSL
3 AlpacaAnnie 8:24 am Feb 3, 2006
Are different User-agent entries needed for versions of similar robot
3 Ceyx 2:29 am Feb 1, 2006
Robots list service?
Looking for a service provider or subscription service
2 xbase234 1:41 am Feb 1, 2006
How to disallow URLs with parameters?
"/page-name.php?" or "/page-name.php"
11 iProgram 11:34 pm Jan 31, 2006
This OK?
3 TonysDesigns 10:13 pm Jan 30, 2006
block robots from images
robot images
2 gb_dev 9:12 pm Jan 30, 2006
If my website dont have robots.txt
robots.txt
7 rusmart 8:50 pm Jan 30, 2006
Duplicate Content in Robots.txt file
2 palmpal 12:17 am Jan 30, 2006
To allow some robots only
9 GoldenHammer 3:14 pm Jan 28, 2006
robots.txt vs htaccess for control
if the robots.txt is not dependable why not just use htaccess?
6 ccubed99 3:39 am Jan 28, 2006
robots.txt and htaccess
4 agradation 11:17 pm Jan 26, 2006
Typical robots.txt?
Anybody got a same robots.txt?
1 1Lit 6:58 pm Jan 26, 2006
Disallow for sub-domain directory?
7 palmpal 4:18 pm Jan 26, 2006
Blocking old pages
1 Nutter 3:21 pm Jan 26, 2006
Wildcards special syntax for Google
publish Google unswer about special Google syntax in robots.txt plus examle
2 spiral 3:44 am Jan 26, 2006
Spiders finding you against all odds
3 Lyndsay 6:49 pm Jan 23, 2006
yahoo/slurp spidering
We don't want these pages indexed by yahoo
9 latimer 8:28 am Jan 23, 2006