homepage Welcome to WebmasterWorld Guest from 50.19.169.37
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconI think there is a problem
  Robots problem
2AllanS 10:39 am on May 17, 2008 (gmt 0)
newpost thread iconrobots.txt syntax for wildcards?
 
3Marfola 7:15 am on May 15, 2008 (gmt 0)
newpost thread iconHow many links can i place in one page of site map?
 
5bava_seo 2:12 pm on May 10, 2008 (gmt 0)
newpost thread iconwhy use and what is robots.txt
  this maybe stupid question
3mick_simon 4:45 pm on May 7, 2008 (gmt 0)
newpost thread iconHow To stop crawler to crawl https:// pages
 
8vreelmistee1 6:03 am on May 7, 2008 (gmt 0)
newpost thread iconStatus: 404(not found) in Google Webmaster Tools
 
3Boulder90 6:39 pm on May 6, 2008 (gmt 0)
newpost thread iconY! and robots.txt
  Partial ignore
5Staffa 6:36 pm on May 6, 2008 (gmt 0)
newpost thread iconGooglebot is Not Taking Visit to my site
  robots.txt is restricted googlebot to do not visit
4bilalseo 1:54 pm on May 6, 2008 (gmt 0)
newpost thread iconSite Pages are Blocked by Robots.txt
 
4bilalseo 10:03 am on May 6, 2008 (gmt 0)
newpost thread iconrobots.txt and IP addresses
 
11Maciej_Ziemczonek 10:11 pm on May 2, 2008 (gmt 0)
newpost thread iconIs this valid? wildcard in robots
  Disallow: /*/edit.html
3rudyten 1:47 pm on Apr 20, 2008 (gmt 0)
newpost thread iconWill spiders follow links on URL's blocked by robots.txt
 
3deanw 6:34 pm on Apr 16, 2008 (gmt 0)
newpost thread iconSecurity Challenges
 
4pageoneresults 3:49 pm on Apr 2, 2008 (gmt 0)
newpost thread iconDumbest question eva: Do all major SEs support the Google guidelines?
  I am testing mine on G's WebmasterCentral
2walkman 6:42 pm on Mar 26, 2008 (gmt 0)
newpost thread iconIs there any way to block everyone except.
  for Google, Yahoo etc?
2mmiller 9:41 am on Mar 21, 2008 (gmt 0)
newpost thread iconneed to block. and don't know - pls help
  need help with blocking - Malware Ref
4Reilly 7:55 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow to block original + localized pages?
  robots.txt localized
1vanguard2000 6:19 pm on Mar 17, 2008 (gmt 0)
newpost thread iconHow long before bots re-visit after they were told they are not allowed?
 
3boromir27 3:32 am on Mar 17, 2008 (gmt 0)
newpost thread iconInktomi User-Agent blocked
  Inktomi blocked
6cyberdyne 8:36 am on Mar 14, 2008 (gmt 0)
newpost thread iconrobots.txt - Disallow/Allow
 
7eyalkattan 1:14 am on Mar 13, 2008 (gmt 0)
newpost thread iconrobots and redirecting
  i am wondering how a robot would act on a page with a redirect
2melanger 8:56 pm on Mar 8, 2008 (gmt 0)
newpost thread iconUnusual log entries
 
5cyberdyne 6:17 pm on Mar 8, 2008 (gmt 0)
newpost thread iconRedirect robot.txt from http to https
  how to redirect properly.
8clyde4210 2:59 pm on Mar 6, 2008 (gmt 0)
newpost thread iconUh oh, it's happening again
  robots.txt blocked out
2wesg 9:58 pm on Feb 28, 2008 (gmt 0)
newpost thread iconnew badbot: EnaBot/1.0
 
7Achernar 8:31 am on Feb 28, 2008 (gmt 0)
Displaying Page 24 of 72<< Back 1 ... 14 15 16 17 18 19 20 21 22 23 [ 24 ] 25 26 27 28 29 30 31 32 33 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved