Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Stopping spiders from Competitors Analysis Tools... |
4 | ambiorix_rb | 1:21 pm June 1, 2005 | ||||
| SEO / robots.txt question unintentional cross-linking penalty |
1 | wolfgang | 6:27 pm May 31, 2005 | ||||
| Blocking a Sub Domain Directory |
3 | is300 | 8:25 am May 30, 2005 | ||||
| Order allow,deny or deny,allow Not sure which order is best... |
3 | whoisgregg | 9:16 pm May 27, 2005 | ||||
| Search Engine Crawlers |
2 | dayo | 8:39 pm May 26, 2005 | ||||
| Spidering of my code pages Wrong pages being spidered and losing referals |
5 | djfelip | 11:18 pm May 25, 2005 | ||||
| robots4.txt -- I'm confused Confused about what is allowed in this file |
3 | Goose_68 | 8:46 pm May 24, 2005 | ||||
| newbie question http://www.yourdomain.com/robots.txt |
2 | webmz | 3:10 am May 22, 2005 | ||||
| Validation |
18 | Dexie | 4:14 pm May 21, 2005 | ||||
| Can't find the robot.txt where is it? |
27 | mmmwowmmm | 10:56 am May 21, 2005 | ||||
| SonicWALL Firewall Blocks Spiders Settings will block spiders from indexing site. |
26 | EliteWeb | 3:16 pm May 15, 2005 | ||||
| Robots.txt Format Will this work? |
4 | kbba04527 | 6:37 am May 6, 2005 | ||||
| googlebot, robots.txt and spidering Perplexed by lack or spidering of the site |
8 | partnermine | 5:12 am May 6, 2005 | ||||
| meta tags+robots.txt newbie question |
3 | bonkph | 8:16 pm May 5, 2005 | ||||
| Baiduspider - not checking robots.txt & how to block? |
6 | TreeShare | 8:07 am May 5, 2005 | ||||
| robots.txt and subdomain Can I use this file to block a specific subdomain |
4 | Pat1975 | 8:57 am May 4, 2005 | ||||
| google hits robots.txt but does not index anything? am i doing something wrong? |
19 | mbatta | 5:38 am May 4, 2005 | ||||
| Can someone tell me how to ........ |
3 | Lex_Luther | 9:00 pm May 3, 2005 | ||||
| Google - User Agent: * Disallow: / |
4 | wchan07 | 4:42 pm May 3, 2005 | ||||
| mod_rewrite and robots.txt |
2 | jewels | 3:03 pm May 3, 2005 | ||||
| Blocking Yahoo from a specific folder How do I do this? |
2 | Rollo | 2:55 pm May 3, 2005 | ||||
| How to Validate robots.txt and do I even have one? |
13 | neophitemoneymaker | 11:58 pm May 1, 2005 | ||||
| Page indexed in Google despite robots.txt entry How did they find it in the first place? |
23 | pmkpmk | 9:17 pm May 1, 2005 | ||||
| The Great SearchEngineWorld robots.txt Survey #2 is Under Way |
9 | Brett_Tabke | 2:35 pm Apr 29, 2005 | ||||
| No-Follow, allow indexing of a file but not it's links? |
2 | JAB_Creations | 8:32 am Apr 27, 2005 |
