homepage Welcome to WebmasterWorld Guest from 54.161.195.4
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRobot Validator Problem
 
7bunky6767 1:13 pm on Jan 6, 2006 (gmt 0)
newpost thread iconBlocking all URLS to a website using the robots.txt file?
 
2scubajared 10:05 am on Jan 6, 2006 (gmt 0)
newpost thread iconTuring keys
 
2Ejap 5:57 pm on Jan 5, 2006 (gmt 0)
newpost thread iconABC of robots.txt
  Now that will be the best tutorial!
8Monzilla 2:19 pm on Jan 4, 2006 (gmt 0)
newpost thread iconDefine allowable spider times[2] ( 1 2 )
  MSN is going nutz!
31Reno 1:44 pm on Jan 4, 2006 (gmt 0)
newpost thread iconNew empty site beeing spidered
 
2sren 3:08 am on Dec 29, 2005 (gmt 0)
newpost thread iconremove PHP from Google.com
  remove PHP from Google.com
2maimax 7:32 am on Dec 28, 2005 (gmt 0)
newpost thread iconpaging google! robots.txt being ignored!
 
14elvey 7:25 am on Dec 27, 2005 (gmt 0)
newpost thread iconHow to block robots spidering certain pages?
 
4zoltan 6:42 pm on Dec 23, 2005 (gmt 0)
newpost thread iconCorrect Syntax for Blogs in robots.txt
 
3vincevincevince 4:27 pm on Dec 22, 2005 (gmt 0)
newpost thread iconBlocking domain specific links
 
3seasalt 11:33 pm on Dec 21, 2005 (gmt 0)
newpost thread iconGoogle appears to be ignoring robots file
  No change after three months
5Lokutus 9:50 pm on Dec 16, 2005 (gmt 0)
newpost thread iconparameter page to be banned from crawling
  directory.asp?
4idolw 1:18 pm on Dec 15, 2005 (gmt 0)
newpost thread iconAbout the robots tag for Googlebot
  Is it necessary?
2a1call 8:57 pm on Dec 14, 2005 (gmt 0)
newpost thread iconRemove part of your website
  Should I use robots.txt or 301 redirects?
1twist 7:27 pm on Dec 13, 2005 (gmt 0)
newpost thread iconRobots.txt and the Google sandbox
  Does de-indexing put one back into the sandbox?
2Rollo 7:05 pm on Dec 13, 2005 (gmt 0)
newpost thread iconShould I use the "?" or not.
 
2kgormat 3:52 am on Dec 12, 2005 (gmt 0)
newpost thread iconrobots.txt and the law
  do they intersect
2Powdork 10:19 am on Dec 11, 2005 (gmt 0)
newpost thread iconNewbie here require help
  robots.txt
2sim20 11:36 pm on Dec 2, 2005 (gmt 0)
newpost thread icondisallow URL with parameters
  /?id=asdf vs /
5Ratbert 11:18 am on Dec 1, 2005 (gmt 0)
newpost thread iconBan Google from 1 directory
  Ban Google from 1 directory
2dunhill 6:47 am on Nov 30, 2005 (gmt 0)
newpost thread iconYahoo Ignoring robots.txt
  Anyone know how to get Yahoo....
2marketingmagic 8:23 pm on Nov 29, 2005 (gmt 0)
newpost thread iconnoimageindex?
  How do I allow a page to be cached without right clicking allowed?
3NeedALife 7:44 pm on Nov 29, 2005 (gmt 0)
newpost thread iconRobots.txt not functioning!
  Added the file months ago but results still coming up
4nwhorton 6:55 pm on Nov 28, 2005 (gmt 0)
newpost thread iconlist of robots to disallow that actually obey robots.txt
 
1koen 3:32 pm on Nov 28, 2005 (gmt 0)
Displaying Page 41 of 72<< Back 1 ... 31 32 33 34 35 36 37 38 39 40 [ 41 ] 42 43 44 45 46 47 48 49 50 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved