homepage Welcome to WebmasterWorld Guest from 54.197.171.109
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt

Sitemaps, Meta Data, and robots.txt

Forum Charter, Library, Moderated by: goodroi Forum Options: Reset Last Read Pointers

Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
new Subject Messages Started by Last Message
newpost thread iconNeed help removing multiple pages
 
3tintin74 11:00 am on Nov 15, 2005 (gmt 0)
newpost thread iconHelp with Disallow
  Not sure if I got this right...
3BillyS 8:52 pm on Nov 14, 2005 (gmt 0)
newpost thread iconstop Yahoo's crawler but for specific domains
  how to stop yahoo's crawler for specif domains
2nrueda 11:40 pm on Nov 8, 2005 (gmt 0)
newpost thread iconwildcard for googlebot
 
4fzx5v0 4:02 pm on Nov 7, 2005 (gmt 0)
newpost thread iconProposing updates to the robots.txt standard
  It is time for a change
16volatilegx 9:20 pm on Nov 4, 2005 (gmt 0)
newpost thread iconwhat i did wrong?
 
5fluxer 3:45 pm on Nov 4, 2005 (gmt 0)
newpost thread iconprotected ffiles and robots
 
2StickeR 7:48 pm on Nov 3, 2005 (gmt 0)
newpost thread iconExculde php url with vars using robots.txt
  Exclude urls with vars using robots.txt
2alexmart 7:44 pm on Nov 3, 2005 (gmt 0)
newpost thread iconIs Temporary Disallow a Problem Later?
 
6bumpaw 2:10 pm on Nov 3, 2005 (gmt 0)
newpost thread icondisallow problem
 
7Autoweb 8:25 am on Nov 3, 2005 (gmt 0)
newpost thread iconAdded robots.txt - 3 days later rankings gone!
 
11Block19Row13 3:45 pm on Nov 2, 2005 (gmt 0)
newpost thread iconOK now I am baffled - question from new to SEO
 
4Iwrite 12:38 pm on Nov 2, 2005 (gmt 0)
newpost thread iconRobots.text question, disallowing files
 
10mikeD 1:37 am on Nov 2, 2005 (gmt 0)
newpost thread iconcrawl delay in google, msn, yahoo etc.
  crawl delay
4freegk 7:03 pm on Nov 1, 2005 (gmt 0)
newpost thread iconIs robots.txt That necessary?
 
5makzan 7:02 pm on Nov 1, 2005 (gmt 0)
newpost thread iconafter google indexs is it too late to use Disallow
 
2watercrazed 8:51 pm on Oct 30, 2005 (gmt 0)
newpost thread iconExclude by filename, not directory?
 
19joergnw10 7:36 pm on Oct 30, 2005 (gmt 0)
newpost thread iconRobots.txt File to Block from Dynamic URLs
  Trying to eliminate duplicate content...
3BillyS 2:07 am on Oct 28, 2005 (gmt 0)
newpost thread icondisallowing a folder with its own domain name
  subdomain being spidered although it has its own domain name
2chandrika 8:52 pm on Oct 26, 2005 (gmt 0)
newpost thread iconrobots.txt in a folder not root
  will this work?
3fatpeter 6:27 am on Oct 26, 2005 (gmt 0)
newpost thread iconRobots.txt and Query Strings
  Can they block non directory paths?
3soquinn 4:45 am on Oct 26, 2005 (gmt 0)
newpost thread iconSpider and registred access to web
 
5andreabg 8:07 am on Oct 19, 2005 (gmt 0)
newpost thread icongoogle accessing blocked image files
  why does it appear that google is not abiding robots.txt?
3latimer 8:01 am on Oct 19, 2005 (gmt 0)
newpost thread iconDisallow all Directories
 
2Omega1Megs 8:48 pm on Oct 14, 2005 (gmt 0)
newpost thread iconDisallow: line in robots.txt
 
2stefan009 5:33 pm on Oct 14, 2005 (gmt 0)
Displaying Page 42 of 71<< Back 1 ... 32 33 34 35 36 37 38 39 40 41 [ 42 ] 43 44 45 46 47 48 49 50 51 ... 71 Next >>
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved