Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| New spider that is actually good. found a "good" new spider |
1 | jcrimi | 12:10 am Aug 22, 2006 | ||||
| robot.txt problem disallowing pages in home directory |
2 | surfpro203 | 2:23 am Aug 16, 2006 | ||||
| Flash web site - need for robots.txt? I have a web site in Flash - is it necessary to have a robots.txt file? |
2 | Cratima | 1:05 pm Aug 15, 2006 | ||||
| Https? Where'd that come from? Site dropped like a rock |
2 | Dragonlady26 | 1:56 am Aug 12, 2006 | ||||
| Please double check my robots.txt |
1 | lee_sufc | 9:30 am Aug 7, 2006 | ||||
| Robots.txt necessary Is the robots.txt file absolutely necessary |
5 | camchoice | 6:57 pm Aug 6, 2006 | ||||
| 'robot.txt' file to be indexed by SEs?! are there any tricky uses of robot.txt file? |
2 | yuvika | 6:56 pm Aug 6, 2006 | ||||
| Disallow index.html in Robots.txt Is it safe to disallow an index.html file of a website |
3 | pixeladdict | 8:48 pm Aug 4, 2006 | ||||
| disallow every 'filename' except. disallow every 'filename' except... |
2 | decbrad | 4:12 am Aug 4, 2006 | ||||
| Robots.txt Exclusion vs. Nofollow Tag |
3 | JasperSEO | 1:39 am Aug 1, 2006 | ||||
| Keep robots away from site at least two weeks |
1 | zoth | 2:43 pm July 29, 2006 | ||||
| Robots.txt exclusion help Robots.txt exclusion help |
3 | webguy7 | 4:57 am July 22, 2006 | ||||
| using robots.txt is it necessary to exclude .php pages |
6 | deborahbaker | 11:53 am July 13, 2006 | ||||
| Robots exclusion revisted How far have we come with controlling robots? |
3 | pixel_juice | 11:12 pm July 7, 2006 | ||||
| Need to redirect bots from subdomain to main site |
1 | nemo1208 | 3:28 pm July 7, 2006 | ||||
| Help with Spiders Cutting back on the bandwidth |
2 | devnet | 4:56 am July 5, 2006 | ||||
| Is there any harm in not having a robots.txt file at all? |
2 | 1Lit | 10:55 am July 4, 2006 | ||||
| 90% of my traffic is spiders and bots. What to do - |
4 | webpro00801 | 6:41 am July 3, 2006 | ||||
| two duplicate sites .com .net confused on how to handle mirror sites |
4 | jcnh74 | 9:33 pm July 1, 2006 | ||||
| Google 5000 character limit for robots.txt? |
6 | physics | 8:53 pm July 1, 2006 | ||||
| how to block all robots except for 2? possible? |
7 | ionchannels | 12:30 pm June 29, 2006 | ||||
| Re-Indexing Timetable Any guesses as to when the SE's will display our listings again. |
4 | kgormat | 7:42 pm June 24, 2006 | ||||
| Disallow: /#example.html# How would the following be viewed by a spider? |
4 | kgormat | 3:29 pm June 24, 2006 | ||||
| Robot behaviour using the robots meta correctly |
1 | Skylo | 12:47 pm June 20, 2006 | ||||
| Understanding Robot Txt Validator Results I don't know what my results mean. |
4 | Caran | 4:28 pm June 17, 2006 |
