homepage Welcome to WebmasterWorld Guest from 54.145.182.50
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconPrevent Google's "cached" link from appearing
  but still stay in the search results
4digitalv 4:51 pm on Jun 8, 2004 (gmt 0)
newpost thread iconDisallowing specific query strings
 
3duke33 3:05 pm on Jun 8, 2004 (gmt 0)
newpost thread iconIs it a spider?
 
2topher 5:38 am on Jun 8, 2004 (gmt 0)
newpost thread iconPreventing robot access to all subdirectories within a directory
 
2Leonardi 12:16 am on Jun 7, 2004 (gmt 0)
newpost thread iconRobots.txt triggers 404 error
  Robots.txt triggers 404 error
2Nout 11:10 am on May 29, 2004 (gmt 0)
newpost thread iconIs this robots.txt ok?
 
5esllou 12:32 pm on May 26, 2004 (gmt 0)
newpost thread iconRemoving "query-stringed" pages from Google
  And other search engines using robotx.txt
3MatthewHSE 2:23 pm on May 25, 2004 (gmt 0)
newpost thread iconStop Spiders Crawling Message Board Edit
  Cgi Factory Message Board
7mn1dbp 12:36 pm on May 25, 2004 (gmt 0)
newpost thread iconGoogle still lists disallowed pages
  How long do disallowed pages stay in the index?
5jam13 10:56 am on May 23, 2004 (gmt 0)
newpost thread iconlarge robots.txt file
  What size will cause a crawler to choke in the file
7DoppyNL 9:43 pm on May 21, 2004 (gmt 0)
newpost thread iconUsing robot.txt to keep unwanted backlinking
  Is this an effective means
2Daily_Sparring 5:14 pm on May 21, 2004 (gmt 0)
newpost thread iconimportance of a robots.txt file
 
11hughie 12:27 am on May 21, 2004 (gmt 0)
newpost thread iconLogic Error
  Wildcard userAgent last entry in robots.txt file?
2pageoneresults 2:34 pm on May 20, 2004 (gmt 0)
newpost thread iconQuestion about robots meta tag, as well as robots.txt....
 
3xsirrionx 11:48 am on May 20, 2004 (gmt 0)
newpost thread iconHow to validate robots.txt on "https" site?
  Please help me validate my "https" site
2CodeCrunch 12:29 am on May 20, 2004 (gmt 0)
newpost thread iconWild chars *.* allowed in "Disallow" of Robots.txt?
  or anything else like *.txt, etc.
2CodeCrunch 12:26 am on May 20, 2004 (gmt 0)
newpost thread iconrobots.txt restrictions conflictual with site's Terms
  Which of them should prevail?
4cristiscu 6:52 am on May 19, 2004 (gmt 0)
newpost thread iconComments at the end of a line bad style, why?
 
3zachariah 6:52 am on May 18, 2004 (gmt 0)
newpost thread iconSpider Testing
 
6cabowabo 3:44 am on May 13, 2004 (gmt 0)
newpost thread iconMeta Tags & Robots.txt
 
4tcustom 1:57 am on May 13, 2004 (gmt 0)
newpost thread iconWhy is Google still indexing my SWF files?
 
2ichthyous 6:10 am on May 10, 2004 (gmt 0)
newpost thread iconDeny or Not to Deny /images
  Denying /images folder
9quebrada 12:19 am on May 6, 2004 (gmt 0)
newpost thread iconproper way of changing site
 
2jmdb71 5:11 pm on May 1, 2004 (gmt 0)
newpost thread iconHow to get spider to visit parts of site more frequently
  Is this possible?
5Pimpernel 10:35 am on Apr 30, 2004 (gmt 0)
newpost thread iconrobots.txt missing?
 
2zyshen 5:19 pm on Apr 27, 2004 (gmt 0)
Displaying Page 57 of 72<< Back 1 ... 47 48 49 50 51 52 53 54 55 56 [ 57 ] 58 59 60 61 62 63 64 65 66 ... 72 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved