Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| robots.txt file - what program to write it in? I use Frontpage 2002 |
6 | Web_Footed_Newbie | 4:00 pm Jan 29, 2003 | ||||
| Robots.txt - What program to use to write? I use Frontpage 2002 |
4 | Web_Footed_Newbie | 3:34 pm Jan 29, 2003 | ||||
| Potbot not respecting robots.txt |
3 | Scooter24 | 1:59 pm Jan 24, 2003 | ||||
| robots.txt and robots meta tag not mutually exclusive? robots.txt robots meta tag |
4 | Check6 | 1:35 pm Jan 24, 2003 | ||||
| "User-Agent"? ...or, just User-Agent ? |
13 | pendanticist | 3:22 am Jan 22, 2003 | ||||
| robots.txt command to disallow indexing I want to only disallows robots to index a specific directory |
2 | wellnesscafe | 4:50 am Jan 18, 2003 | ||||
| Quick question about robots.txt Excluding dynamic pages |
4 | TheDave | 4:09 am Jan 15, 2003 | ||||
| Why would rogue spiders obey robots.txt? |
2 | irock | 5:29 am Jan 9, 2003 | ||||
| what is robots.txt file? |
3 | antipodes | 5:57 am Jan 4, 2003 | ||||
| Getting ERROR_DISALLOW with Inktomi after adding new robots.txt |
7 | bebox | 1:41 pm Dec 29, 2002 | ||||
| Webmasterworld's Robots.txt A couple of questiosn for Brett |
4 | victor | 5:00 pm Dec 27, 2002 | ||||
| Robots.txt Validator bug? Christian Storm from Turnitin Robot maintainers says my robots.txt is wrong |
3 | scottspence | 11:15 am Dec 17, 2002 | ||||
| How do I create a robot.txt? |
9 | palmpal | 1:35 pm Dec 15, 2002 | ||||
| robots.txt to stop the BAD bots There was a past post with the best robots.txt file? |
11 | Googly | 11:18 pm Dec 12, 2002 | ||||
| Robots.txt file disallow target_blank |
3 | junior | 9:06 am Dec 5, 2002 | ||||
| robots.txt robots.txt |
3 | darkroom | 9:50 am Dec 3, 2002 | ||||
| robots.txt and mod_rewrite, meta robots, etc. |
4 | Trisha | 4:17 am Nov 30, 2002 | ||||
| robot.txt syntax |
7 | jamesf4218 | 9:48 am Nov 25, 2002 | ||||
| Ask Jeeves robots.txt entry what UA should be used |
3 | Kerrin | 4:40 pm Nov 7, 2002 | ||||
| logic check on robots.txt will robots.txt properly disallowing a links page prevent link pop penalty? |
9 | wooden | 6:47 pm Nov 5, 2002 | ||||
| Which robots to exclude |
8 | biggles | 1:56 am Nov 4, 2002 | ||||
| robots.txt validator bug The robots.txt validator accepts malformed files (missing User-agent). |
4 | mortenf | 11:02 pm Nov 3, 2002 | ||||
| What is a Robots.txt? |
4 | Noximus | 3:48 pm Oct 28, 2002 | ||||
| Spider Blocking still a bit unclear |
3 | flexsez | 7:43 pm Oct 23, 2002 | ||||
| Confused about robots.txt need a little help |
2 | Yamagotchi | 3:19 pm Oct 21, 2002 |
