homepage Welcome to WebmasterWorld Guest from 54.167.11.16
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconrobot.txt to remove an alias
  Afraid of duplicated content
2Ulkari 5:48 pm on Jan 8, 2004 (gmt 0)
newpost thread iconSpider Names
  I need the names of some SEs usernames
3The_Young_One 2:47 am on Jan 8, 2004 (gmt 0)
newpost thread iconBlocking Includes (SSI)
  blocking ssi
3direktor1 4:03 pm on Jan 7, 2004 (gmt 0)
newpost thread iconSubdomains
  How to exclude
4gosman 2:54 pm on Jan 6, 2004 (gmt 0)
newpost thread iconDoes the leading / make any difference?
  Sorry, I'm new to robots.txt
2AsleepATheWheel 1:07 am on Jan 6, 2004 (gmt 0)
newpost thread iconfaulty robots.txt?
 
5tenaka 2:13 pm on Jan 3, 2004 (gmt 0)
newpost thread iconOnly those on the list are allowed entrance
  All others OUT
7grnidone 5:29 pm on Jan 2, 2004 (gmt 0)
newpost thread icontxt text for website
  txt
4krikri 5:11 pm on Jan 2, 2004 (gmt 0)
newpost thread iconCheck my robots.txt please
  Don't want to block adsense bot
3conroy 2:10 pm on Jan 2, 2004 (gmt 0)
newpost thread iconHow to ban YahooSeeker?
  Used robots file, but it doesn't obey.
2zf007 3:59 am on Jan 1, 2004 (gmt 0)
newpost thread icondeny a spider from visiting a single page
 
3Michael_Weir 3:08 pm on Dec 29, 2003 (gmt 0)
newpost thread iconDisallow indexing on dynamic files
  Disallow indexing on dynamic files
6premzero 2:58 am on Dec 26, 2003 (gmt 0)
newpost thread iconbanning "vscooter" but allowing "scooter"
 
4keyplyr 9:15 pm on Dec 25, 2003 (gmt 0)
newpost thread iconrobots.txt not showing...
 
4mubarak_zeb 3:37 pm on Dec 19, 2003 (gmt 0)
newpost thread iconGetting 403 Forbidden for robots.txt
  Can anyone help
11GodLikeLotus 4:59 am on Dec 18, 2003 (gmt 0)
newpost thread iconrobots.txt use and hosting provider problem
  robots.txt use and hosting provider problem
2david_cool 6:55 pm on Dec 16, 2003 (gmt 0)
newpost thread iconrobots.txt on phpbb forum
 
2RammsteinNicCage 4:06 pm on Dec 12, 2003 (gmt 0)
newpost thread iconSpiders get / and /robots.txt but no more!
  Is my robots.txt killing me?
6dwilson 8:55 pm on Dec 11, 2003 (gmt 0)
newpost thread icon[404] HTTP Page robots.txt not found 404 error
 
4Strike 12:44 pm on Dec 10, 2003 (gmt 0)
newpost thread iconchanging robots.txt
  replacing robots.txt
2darex 8:24 pm on Dec 9, 2003 (gmt 0)
newpost thread iconCan we prevent a form link from being spidered?
  form href prevent spider indexing
3Frannie 2:32 pm on Dec 9, 2003 (gmt 0)
newpost thread iconRobots and directory hierachies
 
4finlay 5:57 pm on Dec 8, 2003 (gmt 0)
newpost thread iconwildcard disallow for pages with varibles
  would this work for pages ending with?keep= : Disallow: *keep=*?
2The_Rabbit 5:25 pm on Dec 8, 2003 (gmt 0)
newpost thread icondisallowing based on part of a query string
 
3jibbajabba 4:39 pm on Dec 2, 2003 (gmt 0)
newpost thread iconWhois.sc Probe/Bot
  Prevent Snapshot Cache
1CustomFit 3:41 am on Dec 2, 2003 (gmt 0)
Displaying Page 63 of 71<< Back 1 ... 53 54 55 56 57 58 59 60 61 62 [ 63 ] 64 65 66 67 68 69 70 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved