Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| robots.txt help |
2 | miracle | 7:16 pm Nov 8, 2006 | ||||
| Link Checker / Site map generator One that observes robots.txt and meta tags |
5 | fish_eye | 5:33 am Nov 7, 2006 | ||||
| Disallow by ISP Where to get a list of ISPs and the countries they are in? |
1 | youfoundjake | 7:13 pm Nov 3, 2006 | ||||
| Made the mistake of putting sub folders I want allowed disallow everthing but? |
2 | Sunnyvale | 7:07 pm Oct 31, 2006 | ||||
| Google remove url tool for supplemental urls a waste of time? Should I bother removing duplicate supplementals? |
3 | Tonerman | 4:20 pm Oct 30, 2006 | ||||
| allow major engines, block the fleas and to add further control |
5 | youfoundjake | 5:16 pm Oct 26, 2006 | ||||
| Dissallowing Dynamic Content Blocking "index.php?" |
2 | trillianjedi | 6:58 pm Oct 25, 2006 | ||||
| Combining htaccess and robots.txt? |
7 | Lady18 | 11:37 pm Oct 24, 2006 | ||||
| disallowing a 'staging server' |
2 | rden17 | 4:11 pm Oct 24, 2006 | ||||
| .htaccess |
2 | ssgumby | 9:53 pm Oct 21, 2006 | ||||
| cloaking robots.txt to avoid hacking |
4 | idolw | 12:26 pm Oct 18, 2006 | ||||
| How to deal with a robots.txt hack attack Guidelines to mitigate the damage from an attack |
3 | Whitey | 10:17 pm Oct 17, 2006 | ||||
| Mainseek bot this bot is ripping through our site |
2 | ssgumby | 6:36 pm Oct 14, 2006 | ||||
| Google indexing weird links ones with // in them |
4 | netchicken1 | 12:54 am Oct 13, 2006 | ||||
| Please welcome your new moderator: goodroi |
20 | engine | 6:35 am Oct 11, 2006 | ||||
| robots.txt problem and subdirs / wildcards |
2 | disgust | 1:44 pm Oct 10, 2006 | ||||
| Google Sitemaps Robots.txt Question |
2 | Tonerman | 5:31 pm Oct 9, 2006 | ||||
| Naming da' bots Same place, different face |
1 | Reno | 11:58 am Oct 7, 2006 | ||||
| TravelLazerBot Anyone seen this bot before? |
2 | jexx | 7:44 pm Oct 3, 2006 | ||||
| Disallow Coldfusion Pages Only Stop cfm pages from being crawled |
2 | stcrim | 3:18 pm Oct 3, 2006 | ||||
| Removing specific file type |
8 | MisterT | 11:40 pm Sep 28, 2006 | ||||
| Robots - Allowing them rather than disallow |
2 | SanDiegoPaul | 10:30 pm Sep 28, 2006 | ||||
| Robots.txt & meta Robots meta tage or just a robots.txt |
4 | pazang | 7:45 pm Sep 27, 2006 | ||||
| Do excluded pages still count as links-in |
3 | johnlod | 8:14 pm Sep 26, 2006 | ||||
| Order of lines in robots.txt file Can you control a bot with the order of the lines? |
3 | bouncybunny | 3:37 am Sep 23, 2006 |
