Welcome to WebmasterWorld Guest from 54.159.42.234

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
What is wrong with my Robots.txt file?
I'd appreciate help with my Robots.txt file
7 Morthan 3:07 am Feb 22, 2008
Help with blocking URLS
5 jaffstar 7:17 am Feb 18, 2008
301 Redirect and Search Ranking
301 Redirect and Search Ranking
4 bilalseo 6:22 am Feb 18, 2008
No life without robots.txt file
This may be old hat to you but it knocked me for six
18 folkranger 7:17 pm Feb 15, 2008
Robots.txt and iframes
How to disallow iframes using robots.txt?
6 foxtunes 11:00 pm Feb 14, 2008
How to Handle Cname URL
Indexed in Google
7 experienced 7:47 am Feb 12, 2008
Google problem with robots.txt
Webmaster tools isn't helping.
21 wesg 8:16 pm Feb 5, 2008
*/? */index.*? is a must in robots.txt
I see /?widgetid=45 indexed
3 AjiNIMC 8:17 am Jan 30, 2008
Robots.txt (/index.jsp?pg name=ccpm/lob/sublob/pgname&site=a1)
Pattern on parameters
2 Surendra 12:20 am Jan 29, 2008
Denying access to everything but the home page?
Using robots.txt to only allow robot access to the home page
3 monkeylytics 8:00 pm Jan 28, 2008
Control Robots with Meta or robots.txt
3 driller41 12:21 pm Jan 15, 2008
Robots.txt disallowed.
12 g1smd 12:18 am Jan 15, 2008
robots.txt with wildcard?
3 tranceformer 12:39 am Jan 13, 2008
Disallowing with robots.txt
robots.txt, disallow certain urls
3 jim_knopf 12:36 am Jan 13, 2008
Getting correct user agent
getting from access log the correct user agent for robots.txt
2 hessodreamy 9:03 pm Jan 11, 2008
Scrapers checking robots.txt
Spiders arrived on heels of robots.txt deployment
4 timster 1:29 am Jan 4, 2008
404 Errors in Webmaster Console
What to do with these pages?
2 rowtc2 11:35 pm Jan 3, 2008
directory and subdirectory access
allowing access to subdirectory but not main directory
3 Mr_Cool_Dude 6:11 pm Jan 2, 2008
robots.txt question
robots.txt examples
2 jim_knopf 5:43 pm Jan 2, 2008
Reference robots.txt file?
Does anyone have one?
3 mmiller 11:30 pm Dec 25, 2007
how to manage cname in robot
don't was to be indexed
4 experienced 2:10 pm Dec 21, 2007
disallow vs ban in .htaccess
12 joergnw10 11:10 am Dec 16, 2007
How do I prevent the SEs from crawling any links to my site w/HTTPS
I don't want the crawlers to index anything w/ HTTPS protocol
3 ZydoSEO 8:37 pm Dec 9, 2007
yahoo google msn etc.
other searches
1 ComputerHermit 5:56 pm Dec 7, 2007
Proposal for robots.txt To Have Greater Flexibility
12 engine 4:58 pm Nov 30, 2007

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members