Welcome to WebmasterWorld Guest from 54.145.117.164

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Allowing robots to index secure directories in root
13 naomi 2:46 pm Apr 14, 2005
Block certain links
2 liviu 1:13 pm Apr 14, 2005
Alexa bot
Do I NEED robots.txt?
3 TheRookie 11:47 pm Apr 12, 2005
robots.txt
Where does it go?
1 mark1 10:55 pm Apr 10, 2005
log entry
is googlebot coming and going
4 Steve_w 9:00 pm Apr 10, 2005
Disallowing https
Can this be done via robots.txt?
2 yosmc 8:45 pm Apr 9, 2005
mod_rewrite and robots.txt
General discussion on use mod_rewrite with robots.txt
7 Vaddy 4:34 pm Apr 7, 2005
New moderator for robots.txt forum
4 engine 7:15 pm Apr 5, 2005
Subdomains and restricted access Web pages
Search Engine Optimization
2 anshul 7:06 pm Apr 5, 2005
What is: "Googles robots NoCache tag" please?
4 beyondenchanted 8:37 pm Apr 4, 2005
Robot.txt
Where it goes..
3 oresam 5:06 pm Apr 4, 2005
Banning googlebot from individual pages via robots.txt
Meta tags are just not doing the job properly
1 Vimes 8:07 am Mar 31, 2005
Wildcards in robots.txt
2 madmatt69 12:52 am Mar 31, 2005
Blocking all files with a .ASP extension
Is this possible with Robots.txt
1 marketrevolution 6:21 pm Mar 30, 2005
disallowing by script variable
6 wheel 6:30 am Mar 29, 2005
Syntax help please
2 jcambert 3:14 am Mar 29, 2005
Specifically allow one bot to index a page
please check my code ;)
5 RonPK 10:37 am Mar 27, 2005
Bots using robots
Do they really have to?
5 cooldoug 8:54 pm Mar 26, 2005
Image disallow
2 Dexie 1:20 pm Mar 26, 2005
Robots.txt Question
Will this cover it?
4 rover 5:40 pm Mar 25, 2005
Robot.txt woes with my mac
Trouble with robot.txt
4 mkfoto 2:21 am Mar 22, 2005
web logs Primer
Looking for general help/information interpreting web site logs
1 scrgeek 3:05 pm Mar 21, 2005
Robots.txt
Disallowed pages listed
4 unperturbed 7:28 pm Mar 20, 2005
robots.txt confusion
question about where to put robots.txt
10 SplitPersona 3:13 pm Mar 17, 2005
exclude vb with robots
1 johndk 9:55 am Mar 16, 2005

Featured Threads

Hot Threads This Week

Hot Threads This Month