homepage Welcome to WebmasterWorld Guest from 54.211.231.221
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconCheck my robots.txt please
  Don't want to block adsense bot
3conroy 2:10 pm on Jan 2, 2004 (gmt 0)
newpost thread iconHow to ban YahooSeeker?
  Used robots file, but it doesn't obey.
2zf007 3:59 am on Jan 1, 2004 (gmt 0)
newpost thread icondeny a spider from visiting a single page
 
3Michael_Weir 3:08 pm on Dec 29, 2003 (gmt 0)
newpost thread iconDisallow indexing on dynamic files
  Disallow indexing on dynamic files
6premzero 2:58 am on Dec 26, 2003 (gmt 0)
newpost thread iconbanning "vscooter" but allowing "scooter"
 
4keyplyr 9:15 pm on Dec 25, 2003 (gmt 0)
newpost thread iconrobots.txt not showing...
 
4mubarak_zeb 3:37 pm on Dec 19, 2003 (gmt 0)
newpost thread iconGetting 403 Forbidden for robots.txt
  Can anyone help
11GodLikeLotus 4:59 am on Dec 18, 2003 (gmt 0)
newpost thread iconrobots.txt use and hosting provider problem
  robots.txt use and hosting provider problem
2david_cool 6:55 pm on Dec 16, 2003 (gmt 0)
newpost thread iconrobots.txt on phpbb forum
 
2RammsteinNicCage 4:06 pm on Dec 12, 2003 (gmt 0)
newpost thread iconSpiders get / and /robots.txt but no more!
  Is my robots.txt killing me?
6dwilson 8:55 pm on Dec 11, 2003 (gmt 0)
newpost thread icon[404] HTTP Page robots.txt not found 404 error
 
4Strike 12:44 pm on Dec 10, 2003 (gmt 0)
newpost thread iconchanging robots.txt
  replacing robots.txt
2darex 8:24 pm on Dec 9, 2003 (gmt 0)
newpost thread iconCan we prevent a form link from being spidered?
  form href prevent spider indexing
3Frannie 2:32 pm on Dec 9, 2003 (gmt 0)
newpost thread iconRobots and directory hierachies
 
4finlay 5:57 pm on Dec 8, 2003 (gmt 0)
newpost thread iconwildcard disallow for pages with varibles
  would this work for pages ending with?keep= : Disallow: *keep=*?
2The_Rabbit 5:25 pm on Dec 8, 2003 (gmt 0)
newpost thread icondisallowing based on part of a query string
 
3jibbajabba 4:39 pm on Dec 2, 2003 (gmt 0)
newpost thread iconWhois.sc Probe/Bot
  Prevent Snapshot Cache
1CustomFit 3:41 am on Dec 2, 2003 (gmt 0)
newpost thread iconHiding robots.txt
 
21keyplyr 1:45 am on Dec 2, 2003 (gmt 0)
newpost thread iconDid you re-test?
 
3hybridx 2:51 pm on Nov 28, 2003 (gmt 0)
newpost thread iconWeird Code Return from Googlebot when robots.txt was spidered
 
35x54u 9:14 pm on Nov 25, 2003 (gmt 0)
newpost thread iconProblem with robots.txt
 
4Nisse 8:08 am on Nov 25, 2003 (gmt 0)
newpost thread iconrobots and dynamic url
  How to ban robots from a dynamic url
3Ledfish 8:09 pm on Nov 23, 2003 (gmt 0)
newpost thread iconGooglebot Wildcard
 
5payday 11:26 am on Nov 22, 2003 (gmt 0)
newpost thread iconImportance of Hotbot
 
5slobizman 9:08 am on Nov 20, 2003 (gmt 0)
newpost thread iconHow long until robots.txt works?
 
11slobizman 3:23 pm on Nov 18, 2003 (gmt 0)
Displaying Page 63 of 71<< Back 1 ... 53 54 55 56 57 58 59 60 61 62 [ 63 ] 64 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved