Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| meta name = robot What does that do? |
7 | aroye | 5:57 pm June 30, 2003 | ||||
| Scooter 3.3 ignores robots.txt? this is ridiculous |
6 | berli | 2:39 am June 27, 2003 | ||||
| Robots.txt Help please mine is missing? |
3 | peteredmonds | 8:37 pm June 25, 2003 | ||||
| Robots.txt Huh? |
2 | zoobie | 6:52 pm June 19, 2003 | ||||
| "robots" What should it be set at? |
4 | kimderella86 | 6:22 pm June 18, 2003 | ||||
| Robots.txt exclusion for dynamic page |
2 | rogerd | 9:22 pm June 17, 2003 | ||||
| Robots.txt & How your site is spidered Positives of Robots.txt file aside from disallowing bots |
4 | floridadesigns | 11:31 pm June 14, 2003 | ||||
| Why should I obey robots.txt Developer questions |
22 | PWalker | 1:01 am June 13, 2003 | ||||
| Php robots.txt parser script does that exitst? if not regex help needed. |
5 | ikbenhet1 | 2:57 pm June 9, 2003 | ||||
| files requested by robots.txt i would like to find out about the files requested by robots |
8 | andrea_edwards | 7:53 pm June 8, 2003 | ||||
| Toolbar and Robots.txt |
8 | rintrah | 8:41 pm June 5, 2003 | ||||
| Robots.txt - files in folders |
3 | rintrah | 9:43 pm June 4, 2003 | ||||
| robots.txt bad spider |
6 | copongcopong | 9:11 pm June 4, 2003 | ||||
| Robots.txt - What's the Point? |
7 | rintrah | 6:59 pm May 30, 2003 | ||||
| Pragma & Cache-Control Are these Safe with Search Engines |
9 | stuart | 8:32 pm May 27, 2003 | ||||
| Quick robots.txt question Do I just need to put the file name |
2 | bateman_ap | 8:21 pm May 24, 2003 | ||||
| Using Allow: / in robots.txt Google says to use it? |
5 | werty | 3:28 am May 20, 2003 | ||||
| How often should a robot check the robots.txt file? Your opinions please |
8 | jrobbio | 2:25 am May 18, 2003 | ||||
| Using Robot.txt to Block Spidering of Link pages Is this a common underhanded practice? |
12 | jdancing | 4:26 pm May 8, 2003 | ||||
| robots.txt should i even bother if i want the whole site indexed? |
9 | madeonmoon | 4:37 pm May 7, 2003 | ||||
| robots.txt used for site with 2 urls |
3 | spinweb | 2:11 am May 7, 2003 | ||||
| robots.txt validator warning suggestion Missing newlines can cause problems, but in some cases can be detected. |
2 | mortenf | 4:42 pm Apr 25, 2003 | ||||
| Dangers of the robots.txt file A warning to the amateurs like myself |
3 | joeuz | 6:37 am Apr 23, 2003 | ||||
| Robots.txt Stopping robots visiting single dynamic page. |
2 | donovana | 2:02 am Apr 22, 2003 | ||||
| Do search engines cache robots.txt? |
10 | trillianjedi | 11:18 pm Apr 16, 2003 |
