Welcome to WebmasterWorld Guest from 22.214.171.124
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Google not obeying robots.txt
Disallow: /*? not going through
|4||smallcompany||4:48 am Sep 26, 2009|
|Successful 301 Redirection?
Switching a website to an entirely new technology, URLS, etc
|5||supermanu||2:39 am Sep 23, 2009|
|Is Pinging Google a crime ?
||5||kinley||8:46 am Sep 20, 2009|
||7||seomonster||4:11 am Sep 15, 2009|
Include Robots.txt in my site
|2||ravikant||12:00 pm Sep 11, 2009|
|Robots.txt disallow everything in folder, but not folder itself
||4||sequence||3:34 pm Sep 10, 2009|
|Procedure for submitting news sitemap
news sitemap procedure
|2||immanuelruby||11:41 am Sep 3, 2009|
|will I be penalised if an affiliate copies my website articles?
||3||kads||12:04 am Sep 2, 2009|
|Sitemap showing less than 25% URLs indexed
||4||dailypress||5:13 pm Aug 31, 2009|
||2||bobbonew||3:27 pm Aug 27, 2009|
|Google not indexing my sites any more
||5||Hans0||12:44 pm Aug 27, 2009|
|Robots.txt - Disallow: or Disallow: /
||16||phred||12:26 pm Aug 27, 2009|
|Different information shown in Different Search Engines
||2||raj1094||11:47 am Aug 27, 2009|
|Sitemaps to Google, Yahoo and Bing, am I missing another one?
||5||silverbytes||8:38 am Aug 22, 2009|
|Sitemap according to Google guidelines
Sitemap Changes lead to Errors
|3||ahdtravel||4:43 am Aug 22, 2009|
|Entries to robots.txt added automatically? Help
||3||Potsie88||6:47 pm Aug 19, 2009|
|Need Sitemap Software that Uses Metadata Tags
search engine sitemap metadata priority tag
|3||quack||4:21 pm Aug 17, 2009|
|Correctly disallowing only parts of a folder
||2||Trax||9:50 pm Aug 16, 2009|
|HTML sitemaps for very large sites
||3||DiscoStu||6:55 pm Aug 10, 2009|
|robots.txt for directories with same name
||3||vero||8:25 pm Aug 5, 2009|
|Joomla robot.txt file - blocked image directory
my horrible mistake
|2||Yangtze||2:31 pm Aug 5, 2009|
|How to create sitemap xml's if I have More than 50,000 urls?
create sitemap xml
|3||ahdtravel||1:45 pm Jul 30, 2009|
|Please, help with robots.txt
||3||abrodski||2:39 pm Jul 29, 2009|
||2||SwipeTheMagnets||12:54 pm Jul 28, 2009|
|Robots.txt: Overriding Disallow?
Is it possible to override a disallow on a directory on a url level?
|3||thatoneguy||6:11 pm Jul 26, 2009|