Welcome to WebmasterWorld Guest from 54.145.213.148

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Correct use of meta data keywords
Correct use of meta data keywords
7 xshadow 7:02 am Sep 11, 2011
Google indexing blocked content
6 fatpeter 5:52 pm Sep 10, 2011
Category and Tag pages in Sitemap
listing noindex pages in sitemap
2 cloud36 6:54 am Sep 9, 2011
Robots.txt for Wordpress and Vbulletin together - Help
Robots.txt for Wordpress and Vbulletin - Help
7 anticipation 11:17 pm Sep 3, 2011
Google's Search Result Display Pattern
5 webexpert2u 3:42 am Aug 31, 2011
robots.txt disallow searching reoccuring folders
6 jchristi 2:14 am Aug 30, 2011
how to disallow feed in robot.txt readable by all bots
5 bhavana 4:42 am Aug 17, 2011
Sitemap for out of stock products
2 sam222 6:43 am Aug 9, 2011
what we do for our robots
6 lucy24 2:22 am Aug 7, 2011
Question on Sitelinks
sitelinks
3 sledge81 10:26 pm Aug 2, 2011
How can I block these kinds of urls?
3 starchild 1:45 am Aug 2, 2011
sitemap for mobile versions
2 sam222 2:11 am Jul 30, 2011
Free tools for creating a sitemap
11 Fotiman 12:13 pm Jul 27, 2011
Blocking Spiders
How to block spiders from crawling your site
6 Brian0275 9:15 pm Jul 25, 2011
which part of "disallow" did you not understand?
9 lucy24 2:46 am Jul 25, 2011
www, www21, wwww, etc. Displaying Same Content & Indexed By SEs
What to do?
5 getxb 6:48 am Jul 22, 2011
How to manage this in robots.txt
disallow a folder but include one page
10 pankajj 1:36 pm Jul 21, 2011
Disallowed Googlebot-Image bot still spidering
7 cyberdyne 8:47 am Jul 9, 2011
Archive.org
for sites not crawled since 2006
8 Neil_McRae 6:04 pm Jul 1, 2011
Googlebot not respecting custom crawl rate
13 rowan194 3:38 am Jun 22, 2011
sitemap for images on subdomain
4 AlexB77 6:50 am Jun 11, 2011
What's the difference between bing bot and Microsoft?
Microsoft IP staying too long...banned for now, what's with that?
3 erlandc 10:21 pm Jun 4, 2011
Google, Bing and Yahoo agree on Microformats?
Was that discussed here: schema.org?
5 pontifex 11:00 pm Jun 3, 2011
robots.txt wildcard syntax?
is this syntax ok?
5 maha 9:39 pm May 31, 2011
Index Page Removal, Serious Problem Please Help
robots.txt or whats the issue? non-technical person here
6 faust1 10:25 am May 29, 2011

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members