Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Site hacked and robots.txt placed, taken off google. robots.txt, hacked, taken off google |
3 | Mohammed | 10:43 am Apr 15, 2003 | ||||
| robots.txt Disallow other URLs for one site. |
3 | spinweb | 6:09 pm Apr 11, 2003 | ||||
| Question about simple robots.txt file |
13 | nadsab | 12:56 pm Apr 10, 2003 | ||||
| Robots.txt Is this valid? |
5 | Alternative_Future | 9:52 am Apr 3, 2003 | ||||
| Get a Robot Get the robot file |
7 | Glenn | 2:45 pm Apr 2, 2003 | ||||
| robots.txt question |
2 | deltron23 | 9:01 pm Mar 28, 2003 | ||||
| External Javascript and bots and robots.txt? |
2 | JeremyL | 6:07 pm Mar 12, 2003 | ||||
| Robot.txt and Templates The <meta>Robot tag in templates, with specific disallow in robot.txt |
5 | juniperwasting | 4:12 pm Mar 11, 2003 | ||||
| My Robots.txt Screw Up. case study in how to kill well performing serps in 1 easy step |
5 | webdevsf | 1:45 am Mar 8, 2003 | ||||
| How do I apply the robots.txt? Seems important and need to know more! |
3 | askjoe | 6:24 pm Mar 7, 2003 | ||||
| I don't want to be listed yet Is disabling anonomas access enough? |
8 | currington | 9:53 am Mar 5, 2003 | ||||
| empty robots.txt yes or no |
6 | HuhuFruFru | 8:32 am Mar 5, 2003 | ||||
| question about an empty robots.txt |
5 | HuhuFruFru | 8:36 pm Mar 4, 2003 | ||||
| logs and robots.txt |
7 | tyrojds | 7:42 pm Mar 4, 2003 | ||||
| Why robots.txt if you want everything indexed? Does it even matter? |
9 | Larry_C | 3:49 pm Mar 3, 2003 | ||||
| Robots.txt question excluding every folder expect one |
5 | JVB_Mktg | 7:54 pm Mar 2, 2003 | ||||
| Logs show a code 206 with robots.txt code 206 - partial content |
2 | nativenewyorker | 11:03 pm Mar 1, 2003 | ||||
| robots.txt syntax spaces or slashes? |
3 | keyplyr | 5:14 pm Feb 27, 2003 | ||||
| Strange Incident with robots.txt error code 401 |
3 | neh2008 | 4:57 pm Feb 17, 2003 | ||||
| robots.txt to block bots access to images got to save bandwidth |
4 | stavs | 3:00 pm Feb 14, 2003 | ||||
| ROBOTS.TXT Location ROBOTS.TXT Location |
4 | neh2008 | 5:27 pm Feb 11, 2003 | ||||
| Robots.txt I better get this right... |
6 | JVB_Mktg | 4:19 am Feb 11, 2003 | ||||
| Robots.txt How? Sub-Domains |
2 | MarkHutch | 8:37 pm Feb 6, 2003 | ||||
| frames and robots.txt |
4 | snowfishin | 9:15 pm Feb 3, 2003 | ||||
| Do proxies and Internet caches respect robots.txt? |
9 | Scooter24 | 12:06 am Feb 3, 2003 |
