Welcome to WebmasterWorld Guest from 22.214.171.124
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
Include Robots.txt in my site
|2||ravikant||12:00 pm Sep 11, 2009|
|Robots.txt disallow everything in folder, but not folder itself
||4||sequence||3:34 pm Sep 10, 2009|
|Procedure for submitting news sitemap
news sitemap procedure
|2||immanuelruby||11:41 am Sep 3, 2009|
|will I be penalised if an affiliate copies my website articles?
||3||kads||12:04 am Sep 2, 2009|
|Sitemap showing less than 25% URLs indexed
||4||dailypress||5:13 pm Aug 31, 2009|
||2||bobbonew||3:27 pm Aug 27, 2009|
|Google not indexing my sites any more
||5||Hans0||12:44 pm Aug 27, 2009|
|Robots.txt - Disallow: or Disallow: /
||16||phred||12:26 pm Aug 27, 2009|
|Different information shown in Different Search Engines
||2||raj1094||11:47 am Aug 27, 2009|
|Sitemaps to Google, Yahoo and Bing, am I missing another one?
||5||silverbytes||8:38 am Aug 22, 2009|
|Sitemap according to Google guidelines
Sitemap Changes lead to Errors
|3||ahdtravel||4:43 am Aug 22, 2009|
|Entries to robots.txt added automatically? Help
||3||Potsie88||6:47 pm Aug 19, 2009|
|Need Sitemap Software that Uses Metadata Tags
search engine sitemap metadata priority tag
|3||quack||4:21 pm Aug 17, 2009|
|Correctly disallowing only parts of a folder
||2||Trax||9:50 pm Aug 16, 2009|
|HTML sitemaps for very large sites
||3||DiscoStu||6:55 pm Aug 10, 2009|
|robots.txt for directories with same name
||3||vero||8:25 pm Aug 5, 2009|
|Joomla robot.txt file - blocked image directory
my horrible mistake
|2||Yangtze||2:31 pm Aug 5, 2009|
|How to create sitemap xml's if I have More than 50,000 urls?
create sitemap xml
|3||ahdtravel||1:45 pm Jul 30, 2009|
|Please, help with robots.txt
||3||abrodski||2:39 pm Jul 29, 2009|
||2||SwipeTheMagnets||12:54 pm Jul 28, 2009|
|Robots.txt: Overriding Disallow?
Is it possible to override a disallow on a directory on a url level?
|3||thatoneguy||6:11 pm Jul 26, 2009|
|Using site maps to circumvent country selection page
||2||hansondodge||2:32 pm Jul 24, 2009|
|1 robots.txt across multiple domains, pt. 2
Continuation of 1 robots.txt file across multiple domains
|2||EvanDonovan||8:15 pm Jul 22, 2009|
|Proper procedure for robot.txt
||2||ahdtravel||8:11 am Jul 20, 2009|
|robot.txt: Disallow every subdirectory? Allow certain subdirectories?
||3||SwipeTheMagnets||2:24 am Jul 20, 2009|