Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Robots.txt file How do I do this... |
2 | lizzie | 8:27 pm May 19, 2001 | ||||
| Robots.txt How effective is it anyway |
6 | circuitjump | 2:35 pm May 11, 2001 | ||||
| Robozilla and Robots.txt |
4 | Brett_Tabke | 8:11 am May 5, 2001 | ||||
| replacing robots.txt with a CGI |
7 | theperlyking | 7:42 pm Apr 25, 2001 | ||||
| robots.txt warning; robots.txt containing DOS or Mac line |
5 | roma | 10:35 am Apr 25, 2001 | ||||
| Robots.txt Do you use it if you want to all all robots? |
7 | WebSeeker | 8:30 am Apr 16, 2001 | ||||
| Looking for complicated borderline Robots.txt |
4 | Brett_Tabke | 10:14 am Apr 9, 2001 | ||||
| Google Link points and Robots.txt Google points algo and side discussion of robots.txt format[2] ( 1 2 ) |
34 | tmw | 2:53 pm Mar 27, 2001 | ||||
| Doorway / Hallway pages Vs. Robots.txt Does one need to build "many" sites under one roof? |
5 | Garyh | 10:45 pm Mar 20, 2001 | ||||
| Yahoo and Robots.txt |
2 | Brett_Tabke | 9:16 pm Feb 28, 2001 | ||||
| How does robot.txt work? |
4 | Acternaweb | 2:59 am Feb 28, 2001 | ||||
| Mr. Roboto How do I set up a Robots.txt file? |
6 | CaveToad | 3:57 pm Jan 31, 2001 | ||||
| Robots.txt and subdomains. How do spiders react on robots.txt on subdomains? |
1 | Rumbas | 11:05 am Jan 29, 2001 | ||||
| Spider.txt |
10 | graemel | 10:47 pm Jan 24, 2001 | ||||
| Spiders Ignoring Robots.txt Content Vs. Legal Link or Site Trespassing. |
6 | Garyh | 4:12 pm Jan 22, 2001 | ||||
| Why would I use a Robots.txt file? |
7 | BoneHeadicus | 12:40 am Jan 14, 2001 | ||||
| Should I be using a Robots.txt file?\n |
6 | BoneHeadicus | 7:40 pm Jan 10, 2001 | ||||
| To "Allow" or not to "Allow" in robots.txt Can it force a deep crawl? |
4 | webtamers | 9:53 am Nov 6, 2000 | ||||
| SEs Check robots.txt and just go away? |
9 | webtamers | 6:14 am Nov 6, 2000 | ||||
| Aim-pro Newsletter: robots.txt and too many urls |
6 | Brett_Tabke | 6:22 pm Jun 29, 2000 |
