homepage Welcome to WebmasterWorld Guest from 107.21.163.227
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconMoved: BPImage walker moved
  Post was moved here: http://www.webmasterworld.com/search_engine_spiders/4503671.htm
1System 8:29 pm on Oct 3, 2012 (gmt 0)
newpost thread iconHow to stop google from indexing wordpress blog paginated archive
 
14Abhinav 6:57 am on Oct 3, 2012 (gmt 0)
newpost thread iconHow to block duplicate pages via robots.txt?
 
5hyderali 6:53 am on Oct 3, 2012 (gmt 0)
newpost thread iconHow do I block the homepage ONLY?
  Perhaps an unusual request
81script 6:22 am on Sep 28, 2012 (gmt 0)
newpost thread iconWho to create XML sitemap exclude URL parameter in Xenu's Link Sleuth
  Best practices to using Xenu's Link Sleuth
3rajesh_magar 10:46 am on Sep 25, 2012 (gmt 0)
newpost thread icondisable indexing only for part of the one page
  disabling part of the page to be indexed
1execom99 1:39 pm on Sep 19, 2012 (gmt 0)
newpost thread iconIncorrect Robots.txt URL in server Logs
 
5triggerfinger 7:57 pm on Sep 17, 2012 (gmt 0)
newpost thread iconRobots.txt Was Inaccessible
  Crawl Postponed Because Robots.txt Was Inaccessible
7poojabhatt 4:21 pm on Sep 16, 2012 (gmt 0)
newpost thread iconMoved: In page analytics moved
  Post was moved here: http://www.webmasterworld.com/analytics/4491502.htm
1System 11:25 am on Sep 4, 2012 (gmt 0)
newpost thread iconGoogle Index robots.txt
  google describes the result for the pages blcocked by robots.txt
9webseosolution 5:41 am on Aug 30, 2012 (gmt 0)
newpost thread iconGoogle appears to be ignoring 301'd URLs
  Googlebot indexing site by IP, even though server is set up to 301
13rowan194 6:50 pm on Aug 28, 2012 (gmt 0)
newpost thread iconBlocking sub-domain in robots.txt?
 
5shaunm 10:24 am on Aug 23, 2012 (gmt 0)
newpost thread iconApache: execute "global" php script when robots.txt requested
  Is there a way to make Apache run a script for a load of a particular file?
10rowan194 7:53 pm on Aug 20, 2012 (gmt 0)
newpost thread iconHow do I block a sub-domain?
  How do I block a dub-domain if the sub-domain is a virtual directory?
6shaunm 10:38 am on Aug 17, 2012 (gmt 0)
newpost thread iconDifferent search result for searching single and plural word
 
2bubble 2:43 pm on Aug 13, 2012 (gmt 0)
newpost thread iconGooglebot possibly ignoring robots.txt
 
23Andem 6:27 am on Jul 29, 2012 (gmt 0)
newpost thread iconNoindex vs Disallow within a Sitemap
  What is the use of using Noindex within a Sitemap?
9shaunm 5:28 am on Jul 27, 2012 (gmt 0)
newpost thread icongoogle duplicate title issue
 
7jon22 10:02 am on Jul 6, 2012 (gmt 0)
newpost thread iconccTLDs and Sitemaps
  Can I have one XML sitemap for all my ccTLDs?
3shaunm 9:35 am on Jun 25, 2012 (gmt 0)
newpost thread iconHTTP 320 URLs, robots.txt and Google
  I need to remove indexed HTTP 302 Redirect URLs from Google index.
5nabucosound 12:38 am on Jun 16, 2012 (gmt 0)
newpost thread iconRobots Follow Meta Tag
  Will nofollow attribute override robots follow meta tag.
4Hoegaarden 1:20 am on Jun 9, 2012 (gmt 0)
newpost thread iconblock ? with robots.txt or htaccess
 
3zeus 10:52 pm on Jun 8, 2012 (gmt 0)
newpost thread iconSitemap Problem: Google shows sitemap URL in results instead of post U
 
2Lemans 10:44 pm on Jun 8, 2012 (gmt 0)
newpost thread iconRobots.txt and regular expressions
  Can they use them?
10jqwan 7:48 pm on Jun 4, 2012 (gmt 0)
newpost thread iconMozilla/4.* (Compatible
  to block or not?
13Seedy 9:25 pm on May 21, 2012 (gmt 0)
Displaying Page 4 of 71<< Back 1 2 3 [ 4 ] 5 6 7 8 9 10 11 12 13 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved