Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| How to stop Yahoo Slurp? Slurp ingnores my robots.txt |
16 | atlrus | 9:47 pm Apr 26, 2005 | ||||
| How to remove ezboard forum from Wayback? |
3 | wmw99 | 12:04 pm Apr 26, 2005 | ||||
| Is there a definitive robots.txt to disallow bad bots bad bots and bad bots only |
6 | Clark | 11:40 am Apr 24, 2005 | ||||
| Disallow keyword Ignored? Who guarantees me that disallow keyword isn't ignored |
5 | bkausbk | 9:02 am Apr 24, 2005 | ||||
| Allowing and disallowing spiders Easy one - can somebody confirm my understanding? |
3 | Zeberdy | 9:19 am Apr 21, 2005 | ||||
| How to disallow getlastpost |
2 | annelisabeth | 9:19 am Apr 21, 2005 | ||||
| Robots.txt Help request. I need help with stopping spyder robots. |
10 | westmore | 6:22 am Apr 21, 2005 | ||||
| Meta Descriptions How many can I have? |
8 | Andromeda | 8:07 pm Apr 20, 2005 | ||||
| Robots.txt Review I need a confirmation for robots.txt from you experts |
11 | RaniaD | 11:51 am Apr 20, 2005 | ||||
| I want to disallow extra for Slurp is this syntax right? |
2 | baze22 | 12:55 pm Apr 19, 2005 | ||||
| Robots.txt vs Robots META tag Does one over-rule the other? |
3 | obsos | 7:14 am Apr 18, 2005 | ||||
| Validation of robots.txt |
3 | designei | 10:50 am Apr 17, 2005 | ||||
| Help w/ robots.txt Google won't toch it robots.txt google |
2 | JackieB | 11:04 am Apr 16, 2005 | ||||
| What was wrong with this robots.txt if anything |
4 | montenegro | 9:25 am Apr 16, 2005 | ||||
| block cgi bin on a virtual host cgi bin is not in root directory |
3 | Reid | 6:26 am Apr 16, 2005 | ||||
| Block specific words from being indexed |
13 | Jer1024 | 7:34 pm Apr 15, 2005 | ||||
| Allowing robots to index secure directories in root |
13 | naomi | 2:46 pm Apr 14, 2005 | ||||
| Block certain links |
2 | liviu | 1:13 pm Apr 14, 2005 | ||||
| Alexa bot Do I NEED robots.txt? |
3 | TheRookie | 11:47 pm Apr 12, 2005 | ||||
| robots.txt Where does it go? |
1 | mark1 | 10:55 pm Apr 10, 2005 | ||||
| log entry is googlebot coming and going |
4 | Steve_w | 9:00 pm Apr 10, 2005 | ||||
| Disallowing https Can this be done via robots.txt? |
2 | yosmc | 8:45 pm Apr 9, 2005 | ||||
| mod_rewrite and robots.txt General discussion on use mod_rewrite with robots.txt |
7 | Vaddy | 4:34 pm Apr 7, 2005 | ||||
| New moderator for robots.txt forum |
4 | engine | 7:15 pm Apr 5, 2005 | ||||
| Subdomains and restricted access Web pages Search Engine Optimization |
2 | anshul | 7:06 pm Apr 5, 2005 |
