Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| How to Handle Cname URL Indexed in Google |
7 | experienced | 7:47 am Feb 12, 2008 | ||||
| Google problem with robots.txt Webmaster tools isn't helping. |
21 | wesg | 8:16 pm Feb 5, 2008 | ||||
| */? */index.*? is a must in robots.txt I see /?widgetid=45 indexed |
3 | AjiNIMC | 8:17 am Jan 30, 2008 | ||||
| Robots.txt (/index.jsp?pg name=ccpm/lob/sublob/pgname&site=a1) Pattern on parameters |
2 | Surendra | 12:20 am Jan 29, 2008 | ||||
| Denying access to everything but the home page? Using robots.txt to only allow robot access to the home page |
3 | monkeylytics | 8:00 pm Jan 28, 2008 | ||||
| Control Robots with Meta or robots.txt |
3 | driller41 | 12:21 pm Jan 15, 2008 | ||||
| Robots.txt disallowed. |
12 | g1smd | 12:18 am Jan 15, 2008 | ||||
| robots.txt with wildcard? |
3 | tranceformer | 12:39 am Jan 13, 2008 | ||||
| Disallowing with robots.txt robots.txt, disallow certain urls |
3 | jim_knopf | 12:36 am Jan 13, 2008 | ||||
| Getting correct user agent getting from access log the correct user agent for robots.txt |
2 | hessodreamy | 9:03 pm Jan 11, 2008 | ||||
| Scrapers checking robots.txt Spiders arrived on heels of robots.txt deployment |
4 | timster | 1:29 am Jan 4, 2008 | ||||
| 404 Errors in Webmaster Console What to do with these pages? |
2 | rowtc2 | 11:35 pm Jan 3, 2008 | ||||
| directory and subdirectory access allowing access to subdirectory but not main directory |
3 | Mr_Cool_Dude | 6:11 pm Jan 2, 2008 | ||||
| robots.txt question robots.txt examples |
2 | jim_knopf | 5:43 pm Jan 2, 2008 | ||||
| Reference robots.txt file? Does anyone have one? |
3 | mmiller | 11:30 pm Dec 25, 2007 | ||||
| how to manage cname in robot don't was to be indexed |
4 | experienced | 2:10 pm Dec 21, 2007 | ||||
| disallow vs ban in .htaccess |
12 | joergnw10 | 11:10 am Dec 16, 2007 | ||||
| How do I prevent the SEs from crawling any links to my site w/HTTPS I don't want the crawlers to index anything w/ HTTPS protocol |
3 | ZydoSEO | 8:37 pm Dec 9, 2007 | ||||
| yahoo google msn etc. other searches |
1 | ComputerHermit | 5:56 pm Dec 7, 2007 | ||||
| Proposal for robots.txt To Have Greater Flexibility |
12 | engine | 4:58 pm Nov 30, 2007 | ||||
| set crawl rate permanently to faster |
1 | agushalim | 5:39 pm Nov 28, 2007 | ||||
| Wildcards on robots.txt disallows |
11 | MindTwist | 5:59 pm Nov 27, 2007 | ||||
| Site title and description missing in search results I had made some changes in robots.txt file |
2 | ecmedia | 2:12 pm Nov 27, 2007 | ||||
| perfect robot text files good robot text files |
2 | likwidholdings | 12:57 am Nov 27, 2007 | ||||
| Google showing site despite blocking in robots.txt The page is not cached though |
3 | ecmedia | 4:28 pm Nov 20, 2007 |
