homepage Welcome to WebmasterWorld Guest from 54.205.59.78
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Website
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconRemoving pages from Google's cache
  Is it best to allow a "robots" "noarchive" crawl first?
1Phil_Payne 2:05 am on Jan 10, 2006 (gmt 0)
newpost thread iconDisallow ALL folders?
  but allow top level html
6Reno 9:14 pm on Jan 9, 2006 (gmt 0)
newpost thread iconHow to disallow widgets from being indexed
  Order of User-agents
2albatros 8:10 pm on Jan 9, 2006 (gmt 0)
newpost thread iconEmpty line after User-agent:
  Could this do any harm?
1taps 5:31 pm on Jan 7, 2006 (gmt 0)
newpost thread iconRobot Validator Problem
 
7bunky6767 1:13 pm on Jan 6, 2006 (gmt 0)
newpost thread iconBlocking all URLS to a website using the robots.txt file?
 
2scubajared 10:05 am on Jan 6, 2006 (gmt 0)
newpost thread iconTuring keys
 
2Ejap 5:57 pm on Jan 5, 2006 (gmt 0)
newpost thread iconABC of robots.txt
  Now that will be the best tutorial!
8Monzilla 2:19 pm on Jan 4, 2006 (gmt 0)
newpost thread iconDefine allowable spider times[2] ( 1 2 )
  MSN is going nutz!
31Reno 1:44 pm on Jan 4, 2006 (gmt 0)
newpost thread iconNew empty site beeing spidered
 
2sren 3:08 am on Dec 29, 2005 (gmt 0)
newpost thread iconremove PHP from Google.com
  remove PHP from Google.com
2maimax 7:32 am on Dec 28, 2005 (gmt 0)
newpost thread iconpaging google! robots.txt being ignored!
 
14elvey 7:25 am on Dec 27, 2005 (gmt 0)
newpost thread iconHow to block robots spidering certain pages?
 
4zoltan 6:42 pm on Dec 23, 2005 (gmt 0)
newpost thread iconCorrect Syntax for Blogs in robots.txt
 
3vincevincevince 4:27 pm on Dec 22, 2005 (gmt 0)
newpost thread iconBlocking domain specific links
 
3seasalt 11:33 pm on Dec 21, 2005 (gmt 0)
newpost thread iconGoogle appears to be ignoring robots file
  No change after three months
5Lokutus 9:50 pm on Dec 16, 2005 (gmt 0)
newpost thread iconparameter page to be banned from crawling
  directory.asp?
4idolw 1:18 pm on Dec 15, 2005 (gmt 0)
newpost thread iconAbout the robots tag for Googlebot
  Is it necessary?
2a1call 8:57 pm on Dec 14, 2005 (gmt 0)
newpost thread iconRemove part of your website
  Should I use robots.txt or 301 redirects?
1twist 7:27 pm on Dec 13, 2005 (gmt 0)
newpost thread iconRobots.txt and the Google sandbox
  Does de-indexing put one back into the sandbox?
2Rollo 7:05 pm on Dec 13, 2005 (gmt 0)
newpost thread iconShould I use the "?" or not.
 
2kgormat 3:52 am on Dec 12, 2005 (gmt 0)
newpost thread iconrobots.txt and the law
  do they intersect
2Powdork 10:19 am on Dec 11, 2005 (gmt 0)
newpost thread iconNewbie here require help
  robots.txt
2sim20 11:36 pm on Dec 2, 2005 (gmt 0)
newpost thread icondisallow URL with parameters
  /?id=asdf vs /
5Ratbert 11:18 am on Dec 1, 2005 (gmt 0)
newpost thread iconBan Google from 1 directory
  Ban Google from 1 directory
2dunhill 6:47 am on Nov 30, 2005 (gmt 0)
Displaying Page 40 of 71<< Back 1 ... 30 31 32 33 34 35 36 37 38 39 [ 40 ] 41 42 43 44 45 46 47 48 49 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved