Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Disallow a single image? Is it possible to block the search engines from indexing a single image? |
4 | PrattTA1 | 4:59 pm June 26, 2007 | ||||
| Blocking Site Server 3.0 Robot How? |
13 | Matt_Probert | 5:53 am June 23, 2007 | ||||
| A list of american search engines. search engine list. American. |
9 | Michel_Samuel | 12:49 am June 22, 2007 | ||||
| Does this trailing slash mean anything? <meta name="robots" content="noindex" /> |
11 | bouncybunny | 7:10 pm June 20, 2007 | ||||
| Need help with a specific robots.txt question |
4 | Kurgano | 9:18 pm June 12, 2007 | ||||
| about:blank >>>what is this then the page loads? |
4 | twtnyc | 11:44 pm June 8, 2007 | ||||
| Allow rather than Disallow Is it possible? |
5 | Marshall | 10:30 pm June 7, 2007 | ||||
| Why is msnbot ignoring this robot's.txt rule? Wildcard disallow ignored |
3 | bouncybunny | 12:17 pm June 6, 2007 | ||||
| Block all Nutch variants. Possible? |
6 | morags | 9:02 pm June 5, 2007 | ||||
| Use FP 2000 and Want to Keep Goo from Indexing It How? |
7 | ember | 12:19 pm May 31, 2007 | ||||
| Allow google to crawl on only primary domains How to achieve exclusion on multiple domains using obj.conf & robots.txt |
3 | justlonging1 | 12:35 pm May 30, 2007 | ||||
| block all except in robots.txt |
2 | visualscope | 2:26 pm May 28, 2007 | ||||
| What is a bot trap? |
2 | fischermx | 4:28 pm May 26, 2007 | ||||
| Custom rule for google, url args and wordpress feeds |
6 | Kurt | 2:28 pm May 24, 2007 | ||||
| Wordpress - Robots.txt to fix Duplicate Content I need a robots.txt file to prevent dup content from being crawled |
7 | jman123 | 2:16 pm May 23, 2007 | ||||
| Alexa robot issue with alexa bot & alexa toolbar users |
6 | Michel_Samuel | 10:45 am May 20, 2007 | ||||
| Robots.txt Can you exclude a parameter in robots.txt? |
3 | North_Shore | 9:22 pm May 18, 2007 | ||||
| robots.txt mess-up |
4 | strawbleu | 9:12 pm May 18, 2007 | ||||
| XML Stemap |
6 | ajitaryan | 11:35 am May 16, 2007 | ||||
| Is this valid? |
3 | aha7 | 12:34 pm May 9, 2007 | ||||
| Disallowing content previously indexed What happens to indexed content subsequently blocked by robots.txt |
3 | daveozzz | 11:22 am May 9, 2007 | ||||
| duplicate content indexing |
11 | coburn | 6:34 pm May 7, 2007 | ||||
| Denying one page with multiple variables page.html?item=1, page.html?item=2, page.html?item=3 |
7 | wfernley | 6:21 pm May 7, 2007 | ||||
| Can anyone post a complete robots.txt file here? |
4 | PowerUp | 2:29 pm May 7, 2007 | ||||
| How do I disallow bots from my Wordpress year Archive Folders I guess this is a common question but I can't find an answer |
4 | Drum | 5:26 pm May 5, 2007 |
