Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Removing Pages - Does Robots.txt Help? |
3 | akreider | 4:17 am Dec 16, 2006 | ||||
| Allowing only some pages for ia archive |
5 | sosoo | 8:20 am Dec 13, 2006 | ||||
| Mediapartners-Google fellow the robots.txt? |
2 | AlphaWu | 1:41 pm Dec 12, 2006 | ||||
| my robots file need help |
3 | experienced | 12:26 am Dec 12, 2006 | ||||
| Disagreement between Google and a validator Robots.txt Checker says it's so, Google robots anayler says the other way |
6 | Gusgsm | 9:49 am Dec 11, 2006 | ||||
| Googlebot tripping bad bot script |
5 | RobertRogers | 5:29 pm Dec 10, 2006 | ||||
| Is this bad robots.txt |
1 | scoobydooaustralia | 8:02 pm Dec 8, 2006 | ||||
| duplicate pages .aspx?id= Not sure what to do next |
2 | schatzm | 9:01 pm Dec 6, 2006 | ||||
| Is this a bot? Unwanted visitor to my website |
2 | brancook | 8:57 pm Dec 6, 2006 | ||||
| Two domains mirror each other Is it possible to a command to a robots.txt file to apply to just one? |
4 | Jeremy_H | 9:08 pm Dec 5, 2006 | ||||
| Do I need to disallow files that aren't linked to? for example, 'included' PHP & javascript pages |
9 | sssweb | 3:34 pm Dec 4, 2006 | ||||
| removing permanent redirect - honoring content in page removing permanent redirect |
2 | thu_household | 2:56 pm Dec 4, 2006 | ||||
| Do we need robots at all? New member here Confused as to whether I need to add robots txt |
5 | plenusvita | 9:23 pm Dec 2, 2006 | ||||
| Block all spiders, but only allow well known spiders? |
2 | foxfox | 3:19 pm Dec 2, 2006 | ||||
| msnbot visits robots.txt and exits |
7 | pemba76 | 12:13 pm Nov 28, 2006 | ||||
| Upload of incorrect robots.txt What a screw up - any permanent damage? |
4 | TinkyWinky | 10:12 am Nov 28, 2006 | ||||
| Does excluding Googlebot create a penalty? If other bots are allowed, is this a red flag to Google |
2 | AndyA | 9:26 pm Nov 27, 2006 | ||||
| what to put into robots.txt? |
11 | Hitman3266 | 5:09 pm Nov 27, 2006 | ||||
| How long does it take for SEs to learn about a robots'txt |
4 | bouncybunny | 12:04 pm Nov 20, 2006 | ||||
| Yahoo! Slurp Now Supports Wildcards in robots.txt |
17 | OldWolf | 2:54 am Nov 20, 2006 | ||||
| Robots.txt and password-protection |
3 | hwy2hel | 10:56 pm Nov 16, 2006 | ||||
| Yahoo robots txt nedd to disallow all urls with? |
2 | cheaperholidays | 4:54 pm Nov 16, 2006 | ||||
| robots.txt? OR .htaccess? disallowing or protecting directories |
2 | johnjoyce | 4:27 pm Nov 16, 2006 | ||||
| Wildcards in robots.txt for forum software |
6 | Asia_Expat | 1:40 am Nov 14, 2006 | ||||
| robots.txt syntax '$' and '?' |
2 | sssweb | 8:40 pm Nov 11, 2006 |
