homepage Welcome to WebmasterWorld Guest from 54.161.246.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconIs there any bots to block?
  Else than the infamous "Web Content International"?
3Daemon 11:56 pm on Jan 16, 2004 (gmt 0)
newpost thread iconRobots.txt help
 
5mikemcs 1:04 pm on Jan 16, 2004 (gmt 0)
newpost thread iconWho are these spiders and what do they want?
 
7obsos 4:51 pm on Jan 14, 2004 (gmt 0)
newpost thread iconRevisit instruction for robots
  I want to specify different revisit instruction different sections
3adityamodi 5:13 am on Jan 10, 2004 (gmt 0)
newpost thread iconHave I been blocking Altavista spider from spidering my site?
 
6irock 6:10 pm on Jan 9, 2004 (gmt 0)
newpost thread iconrobot.txt to remove an alias
  Afraid of duplicated content
2Ulkari 5:48 pm on Jan 8, 2004 (gmt 0)
newpost thread iconSpider Names
  I need the names of some SEs usernames
3The_Young_One 2:47 am on Jan 8, 2004 (gmt 0)
newpost thread iconBlocking Includes (SSI)
  blocking ssi
3direktor1 4:03 pm on Jan 7, 2004 (gmt 0)
newpost thread iconSubdomains
  How to exclude
4gosman 2:54 pm on Jan 6, 2004 (gmt 0)
newpost thread iconDoes the leading / make any difference?
  Sorry, I'm new to robots.txt
2AsleepATheWheel 1:07 am on Jan 6, 2004 (gmt 0)
newpost thread iconfaulty robots.txt?
 
5tenaka 2:13 pm on Jan 3, 2004 (gmt 0)
newpost thread iconOnly those on the list are allowed entrance
  All others OUT
7grnidone 5:29 pm on Jan 2, 2004 (gmt 0)
newpost thread icontxt text for website
  txt
4krikri 5:11 pm on Jan 2, 2004 (gmt 0)
newpost thread iconCheck my robots.txt please
  Don't want to block adsense bot
3conroy 2:10 pm on Jan 2, 2004 (gmt 0)
newpost thread iconHow to ban YahooSeeker?
  Used robots file, but it doesn't obey.
2zf007 3:59 am on Jan 1, 2004 (gmt 0)
newpost thread icondeny a spider from visiting a single page
 
3Michael_Weir 3:08 pm on Dec 29, 2003 (gmt 0)
newpost thread iconDisallow indexing on dynamic files
  Disallow indexing on dynamic files
6premzero 2:58 am on Dec 26, 2003 (gmt 0)
newpost thread iconbanning "vscooter" but allowing "scooter"
 
4keyplyr 9:15 pm on Dec 25, 2003 (gmt 0)
newpost thread iconrobots.txt not showing...
 
4mubarak_zeb 3:37 pm on Dec 19, 2003 (gmt 0)
newpost thread iconGetting 403 Forbidden for robots.txt
  Can anyone help
11GodLikeLotus 4:59 am on Dec 18, 2003 (gmt 0)
newpost thread iconrobots.txt use and hosting provider problem
  robots.txt use and hosting provider problem
2david_cool 6:55 pm on Dec 16, 2003 (gmt 0)
newpost thread iconrobots.txt on phpbb forum
 
2RammsteinNicCage 4:06 pm on Dec 12, 2003 (gmt 0)
newpost thread iconSpiders get / and /robots.txt but no more!
  Is my robots.txt killing me?
6dwilson 8:55 pm on Dec 11, 2003 (gmt 0)
newpost thread icon[404] HTTP Page robots.txt not found 404 error
 
4Strike 12:44 pm on Dec 10, 2003 (gmt 0)
newpost thread iconchanging robots.txt
  replacing robots.txt
2darex 8:24 pm on Dec 9, 2003 (gmt 0)
Displaying Page 63 of 72<< Back 1 ... 53 54 55 56 57 58 59 60 61 62 [ 63 ] 64 65 66 67 68 69 70 71 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved