homepage Welcome to WebmasterWorld Guest from 54.167.171.79
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconDynamic URL's and robots.txt
 
4cgydosh 3:37 pm on Jul 2, 2008 (gmt 0)
newpost thread iconHow to setup robots.txt?
 
3Cameron08 10:23 am on Jun 27, 2008 (gmt 0)
newpost thread iconIndexing problem for few pages in Google
  I have optimized one static web site vinsystech.com.
3venuakkala 12:49 pm on Jun 24, 2008 (gmt 0)
newpost thread iconSyntax for disallowing folder with '.' in the name?
  Asking for clarification
3ALbino 10:40 pm on Jun 23, 2008 (gmt 0)
newpost thread iconGoogle ignores one line of robots.txt
 
8wheelie34 9:16 am on Jun 22, 2008 (gmt 0)
newpost thread iconGoogle under cover?
 
3Guido 1:38 pm on Jun 13, 2008 (gmt 0)
newpost thread iconNeed to block and remove 1 page from serps
 
4StaceyJ 8:32 pm on Jun 6, 2008 (gmt 0)
newpost thread iconspiders list
 
3hugh 12:49 pm on Jun 6, 2008 (gmt 0)
newpost thread iconHome page blocked accidentally by robots tag
  home page dropped from serps
5jeffe 6:19 pm on May 21, 2008 (gmt 0)
newpost thread iconrobots.txt file being ignored and old content not being removed
 
10Marty3454 11:46 pm on May 20, 2008 (gmt 0)
newpost thread iconTo use or not to use a robots.txt file
 
3con771 6:03 pm on May 20, 2008 (gmt 0)
newpost thread icon4 Wordpress File's Link To Be Blocked
  How should I block them?
5aced84 3:54 pm on May 19, 2008 (gmt 0)
newpost thread iconI think there is a problem
  Robots problem
2AllanS 10:39 am on May 17, 2008 (gmt 0)
newpost thread iconrobots.txt syntax for wildcards?
 
3Marfola 7:15 am on May 15, 2008 (gmt 0)
newpost thread iconHow many links can i place in one page of site map?
 
5bava_seo 2:12 pm on May 10, 2008 (gmt 0)
newpost thread iconwhy use and what is robots.txt
  this maybe stupid question
3mick_simon 4:45 pm on May 7, 2008 (gmt 0)
newpost thread iconHow To stop crawler to crawl https:// pages
 
8vreelmistee1 6:03 am on May 7, 2008 (gmt 0)
newpost thread iconStatus: 404(not found) in Google Webmaster Tools
 
3Boulder90 6:39 pm on May 6, 2008 (gmt 0)
newpost thread iconY! and robots.txt
  Partial ignore
5Staffa 6:36 pm on May 6, 2008 (gmt 0)
newpost thread iconGooglebot is Not Taking Visit to my site
  robots.txt is restricted googlebot to do not visit
4bilalseo 1:54 pm on May 6, 2008 (gmt 0)
newpost thread iconSite Pages are Blocked by Robots.txt
 
4bilalseo 10:03 am on May 6, 2008 (gmt 0)
newpost thread iconrobots.txt and IP addresses
 
11Maciej_Ziemczonek 10:11 pm on May 2, 2008 (gmt 0)
newpost thread iconIs this valid? wildcard in robots
  Disallow: /*/edit.html
3rudyten 1:47 pm on Apr 20, 2008 (gmt 0)
newpost thread iconWill spiders follow links on URL's blocked by robots.txt
 
3deanw 6:34 pm on Apr 16, 2008 (gmt 0)
newpost thread iconSecurity Challenges
 
4pageoneresults 3:49 pm on Apr 2, 2008 (gmt 0)
Displaying Page 24 of 72<< Back 1 ... 14 15 16 17 18 19 20 21 22 23 [ 24 ] 25 26 27 28 29 30 31 32 33 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved