homepage Welcome to WebmasterWorld Guest from 54.167.249.155
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconSpider Names
  I need the names of some SEs usernames
3The_Young_One 2:47 am on Jan 8, 2004 (gmt 0)
newpost thread iconBlocking Includes (SSI)
  blocking ssi
3direktor1 4:03 pm on Jan 7, 2004 (gmt 0)
newpost thread iconSubdomains
  How to exclude
4gosman 2:54 pm on Jan 6, 2004 (gmt 0)
newpost thread iconDoes the leading / make any difference?
  Sorry, I'm new to robots.txt
2AsleepATheWheel 1:07 am on Jan 6, 2004 (gmt 0)
newpost thread iconfaulty robots.txt?
 
5tenaka 2:13 pm on Jan 3, 2004 (gmt 0)
newpost thread iconOnly those on the list are allowed entrance
  All others OUT
7grnidone 5:29 pm on Jan 2, 2004 (gmt 0)
newpost thread icontxt text for website
  txt
4krikri 5:11 pm on Jan 2, 2004 (gmt 0)
newpost thread iconCheck my robots.txt please
  Don't want to block adsense bot
3conroy 2:10 pm on Jan 2, 2004 (gmt 0)
newpost thread iconHow to ban YahooSeeker?
  Used robots file, but it doesn't obey.
2zf007 3:59 am on Jan 1, 2004 (gmt 0)
newpost thread icondeny a spider from visiting a single page
 
3Michael_Weir 3:08 pm on Dec 29, 2003 (gmt 0)
newpost thread iconDisallow indexing on dynamic files
  Disallow indexing on dynamic files
6premzero 2:58 am on Dec 26, 2003 (gmt 0)
newpost thread iconbanning "vscooter" but allowing "scooter"
 
4keyplyr 9:15 pm on Dec 25, 2003 (gmt 0)
newpost thread iconrobots.txt not showing...
 
4mubarak_zeb 3:37 pm on Dec 19, 2003 (gmt 0)
newpost thread iconGetting 403 Forbidden for robots.txt
  Can anyone help
11GodLikeLotus 4:59 am on Dec 18, 2003 (gmt 0)
newpost thread iconrobots.txt use and hosting provider problem
  robots.txt use and hosting provider problem
2david_cool 6:55 pm on Dec 16, 2003 (gmt 0)
newpost thread iconrobots.txt on phpbb forum
 
2RammsteinNicCage 4:06 pm on Dec 12, 2003 (gmt 0)
newpost thread iconSpiders get / and /robots.txt but no more!
  Is my robots.txt killing me?
6dwilson 8:55 pm on Dec 11, 2003 (gmt 0)
newpost thread icon[404] HTTP Page robots.txt not found 404 error
 
4Strike 12:44 pm on Dec 10, 2003 (gmt 0)
newpost thread iconchanging robots.txt
  replacing robots.txt
2darex 8:24 pm on Dec 9, 2003 (gmt 0)
newpost thread iconCan we prevent a form link from being spidered?
  form href prevent spider indexing
3Frannie 2:32 pm on Dec 9, 2003 (gmt 0)
newpost thread iconRobots and directory hierachies
 
4finlay 5:57 pm on Dec 8, 2003 (gmt 0)
newpost thread iconwildcard disallow for pages with varibles
  would this work for pages ending with?keep= : Disallow: *keep=*?
2The_Rabbit 5:25 pm on Dec 8, 2003 (gmt 0)
newpost thread icondisallowing based on part of a query string
 
3jibbajabba 4:39 pm on Dec 2, 2003 (gmt 0)
newpost thread iconWhois.sc Probe/Bot
  Prevent Snapshot Cache
1CustomFit 3:41 am on Dec 2, 2003 (gmt 0)
newpost thread iconHiding robots.txt
 
21keyplyr 1:45 am on Dec 2, 2003 (gmt 0)
Displaying Page 63 of 71<< Back 1 ... 53 54 55 56 57 58 59 60 61 62 [ 63 ] 64 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved