homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconWhat to do with duplicate pages
 
3olba 2:12 am on Jan 12, 2007 (gmt 0)
newpost thread iconWhy should I have a robots.txt file?
  Isn't it OK if I don't?
23beren 8:45 am on Jan 11, 2007 (gmt 0)
newpost thread iconGoogle Site Diagnostics - robots.txt blocks urls
 
4akreider 7:44 pm on Jan 5, 2007 (gmt 0)
newpost thread iconRobots txt to solve gallery duplicates
  can someone confirm this is correct
3caines 10:54 am on Jan 5, 2007 (gmt 0)
newpost thread iconHow many search engine understand crawl delay?
 
1foxfox 3:38 pm on Jan 4, 2007 (gmt 0)
newpost thread iconMultiple Domains Issue
 
3kdobson99 4:58 am on Jan 4, 2007 (gmt 0)
newpost thread iconDo spiders follow links from pages with the noindex tag?
  As opposed to links from robots files?
4bouncybunny 6:59 am on Jan 2, 2007 (gmt 0)
newpost thread iconDirecting Spiders to specific Servers
 
2TheSeoGuy 1:50 pm on Dec 29, 2006 (gmt 0)
newpost thread iconI had robots.txt blocking my phpbb folders which are now deleted
  Should I remove that line from robots.txt now?
2cmendla 4:16 pm on Dec 24, 2006 (gmt 0)
newpost thread iconDisallowing Indexed files in Google
 
5leo11877 2:01 pm on Dec 23, 2006 (gmt 0)
newpost thread iconDisallowing by date.
  Is it possible to keep some page and disallow new ones...?
3bouncybunny 8:31 am on Dec 20, 2006 (gmt 0)
newpost thread iconWorried about profile links
 
4Asia_Expat 3:53 pm on Dec 19, 2006 (gmt 0)
newpost thread iconIs it possible to exclude a URL by specifying an anchor?
  Excluding #...?
3bouncybunny 12:40 am on Dec 17, 2006 (gmt 0)
newpost thread iconRemoving Pages - Does Robots.txt Help?
 
3akreider 4:17 am on Dec 16, 2006 (gmt 0)
newpost thread iconAllowing only some pages for ia archive
 
5sosoo 8:20 am on Dec 13, 2006 (gmt 0)
newpost thread iconMediapartners-Google fellow the robots.txt?
 
2AlphaWu 1:41 pm on Dec 12, 2006 (gmt 0)
newpost thread iconmy robots file
  need help
3experienced 12:26 am on Dec 12, 2006 (gmt 0)
newpost thread iconDisagreement between Google and a validator
  Robots.txt Checker says it's so, Google robots anayler says the other way
6Gusgsm 9:49 am on Dec 11, 2006 (gmt 0)
newpost thread iconGooglebot tripping bad bot script
 
5RobertRogers 5:29 pm on Dec 10, 2006 (gmt 0)
newpost thread iconIs this bad robots.txt
 
1scoobydooaustralia 8:02 pm on Dec 8, 2006 (gmt 0)
newpost thread iconduplicate pages .aspx?id=
  Not sure what to do next
2schatzm 9:01 pm on Dec 6, 2006 (gmt 0)
newpost thread iconIs this a bot?
  Unwanted visitor to my website
2brancook 8:57 pm on Dec 6, 2006 (gmt 0)
newpost thread iconTwo domains mirror each other
  Is it possible to a command to a robots.txt file to apply to just one?
4Jeremy_H 9:08 pm on Dec 5, 2006 (gmt 0)
newpost thread iconDo I need to disallow files that aren't linked to?
  for example, 'included' PHP & javascript pages
9sssweb 3:34 pm on Dec 4, 2006 (gmt 0)
newpost thread iconremoving permanent redirect - honoring content in page
  removing permanent redirect
2thu_household 2:56 pm on Dec 4, 2006 (gmt 0)
Displaying Page 32 of 72<< Back 1 ... 22 23 24 25 26 27 28 29 30 31 [ 32 ] 33 34 35 36 37 38 39 40 41 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved