homepage Welcome to WebmasterWorld Guest from 54.145.243.51
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconspiders list
 
3hugh 12:49 pm on Jun 6, 2008 (gmt 0)
newpost thread iconHome page blocked accidentally by robots tag
  home page dropped from serps
5jeffe 6:19 pm on May 21, 2008 (gmt 0)
newpost thread iconrobots.txt file being ignored and old content not being removed
 
10Marty3454 11:46 pm on May 20, 2008 (gmt 0)
newpost thread iconTo use or not to use a robots.txt file
 
3con771 6:03 pm on May 20, 2008 (gmt 0)
newpost thread icon4 Wordpress File's Link To Be Blocked
  How should I block them?
5aced84 3:54 pm on May 19, 2008 (gmt 0)
newpost thread iconI think there is a problem
  Robots problem
2AllanS 10:39 am on May 17, 2008 (gmt 0)
newpost thread iconrobots.txt syntax for wildcards?
 
3Marfola 7:15 am on May 15, 2008 (gmt 0)
newpost thread iconHow many links can i place in one page of site map?
 
5bava_seo 2:12 pm on May 10, 2008 (gmt 0)
newpost thread iconwhy use and what is robots.txt
  this maybe stupid question
3mick_simon 4:45 pm on May 7, 2008 (gmt 0)
newpost thread iconHow To stop crawler to crawl https:// pages
 
8vreelmistee1 6:03 am on May 7, 2008 (gmt 0)
newpost thread iconStatus: 404(not found) in Google Webmaster Tools
 
3Boulder90 6:39 pm on May 6, 2008 (gmt 0)
newpost thread iconY! and robots.txt
  Partial ignore
5Staffa 6:36 pm on May 6, 2008 (gmt 0)
newpost thread iconGooglebot is Not Taking Visit to my site
  robots.txt is restricted googlebot to do not visit
4bilalseo 1:54 pm on May 6, 2008 (gmt 0)
newpost thread iconSite Pages are Blocked by Robots.txt
 
4bilalseo 10:03 am on May 6, 2008 (gmt 0)
newpost thread iconrobots.txt and IP addresses
 
11Maciej_Ziemczonek 10:11 pm on May 2, 2008 (gmt 0)
newpost thread iconIs this valid? wildcard in robots
  Disallow: /*/edit.html
3rudyten 1:47 pm on Apr 20, 2008 (gmt 0)
newpost thread iconWill spiders follow links on URL's blocked by robots.txt
 
3deanw 6:34 pm on Apr 16, 2008 (gmt 0)
newpost thread iconSecurity Challenges
 
4pageoneresults 3:49 pm on Apr 2, 2008 (gmt 0)
newpost thread iconDumbest question eva: Do all major SEs support the Google guidelines?
  I am testing mine on G's WebmasterCentral
2walkman 6:42 pm on Mar 26, 2008 (gmt 0)
newpost thread iconIs there any way to block everyone except.
  for Google, Yahoo etc?
2mmiller 9:41 am on Mar 21, 2008 (gmt 0)
newpost thread iconneed to block. and don't know - pls help
  need help with blocking - Malware Ref
4Reilly 7:55 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow to block original + localized pages?
  robots.txt localized
1vanguard2000 6:19 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow long before bots re-visit after they were told they are not allowed?
 
3boromir27 3:32 am on Mar 17, 2008 (gmt 0)
newpost thread iconInktomi User-Agent blocked
  Inktomi blocked
6cyberdyne 8:36 am on Mar 14, 2008 (gmt 0)
newpost thread iconrobots.txt - Disallow/Allow
 
7eyalkattan 1:14 am on Mar 13, 2008 (gmt 0)
Displaying Page 24 of 72<< Back 1 ... 14 15 16 17 18 19 20 21 22 23 [ 24 ] 25 26 27 28 29 30 31 32 33 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved