Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Can I Disallow Domain Hosts? I want to disallow spiders from hosts, like foo.mydomain.com |
1 | SoCalSam | 7:24 pm Jan 30, 2005 | ||||
| Robots.txt conundrum! |
4 | energylevel | 6:36 pm Jan 30, 2005 | ||||
| Help I'm trying to get my site on search engines such as google Help on how to get your site on search engines such as google |
9 | Mako3 | 8:32 am Jan 30, 2005 | ||||
| Correct way to exclude dynamic pages question about differing methods of excluding dynamic pages |
8 | heuristick | 6:50 pm Jan 28, 2005 | ||||
| are the robots.txt examples still valid from WebmasterWorld? |
1 | keywordguru | 2:59 pm Jan 28, 2005 | ||||
| Suggestions On 'Low Value' Bots Which bots provide little to no value to the web site. |
2 | netcent | 6:24 pm Jan 24, 2005 | ||||
| Blocking a file in the main folder? |
4 | Ept103 | 3:33 am Jan 23, 2005 | ||||
| robot.txt needed to be crawled by google? bot leaves after not finding one |
8 | thegreatpretender | 4:25 pm Jan 22, 2005 | ||||
| Blocking sherlock quick question to confirm format |
11 | Reno | 2:03 am Jan 20, 2005 | ||||
| Search Engines doesn't work with my robots.txt file |
2 | nirpet | 10:38 pm Jan 18, 2005 | ||||
| google follows links disallowed Have I done something wrong? |
5 | shady | 4:54 pm Jan 18, 2005 | ||||
| Please clarify this for the last time Subfolder, particualr bots exclusion |
1 | damlag | 10:12 pm Jan 11, 2005 | ||||
| exclude subdirectories |
3 | seoerer | 11:08 pm Jan 7, 2005 | ||||
| Robot.txt Generator should i put the results on my website? |
3 | rj87uk | 10:54 am Jan 6, 2005 | ||||
| Not sure how to handle space character |
5 | Jon_King | 1:48 am Jan 6, 2005 | ||||
| Problem of syntax and ordering |
1 | esllou | 11:50 am Jan 4, 2005 | ||||
| Will robots.txt affect IE or other browsers? A little confuse... |
2 | iProgram | 4:34 am Jan 4, 2005 | ||||
| what are the names of the image bots? I can't ban them if I don't know their names |
8 | stapel | 5:17 pm Jan 2, 2005 | ||||
| Blocking images folder from spiders Good or bad idea? |
2 | twist | 10:17 pm Jan 1, 2005 | ||||
| Sudden Increase in robots.txt Hits |
3 | Rob_Moore | 10:04 pm Dec 26, 2004 | ||||
| Keeping spiders off co-branded sites how to keep spiders from indexing our partner / co-branded sites |
4 | BillingOrchard | 9:41 pm Dec 23, 2004 | ||||
| About robots.txt file Confusion in robots.txt file...i need hep.. |
7 | venuc | 5:32 am Dec 21, 2004 | ||||
| Googlebot Spoofing How do you tell the real googlebot from spoofers? |
3 | teaperson | 6:29 pm Dec 20, 2004 | ||||
| Tricky Robots Txt Help, Two sites one robot |
3 | badboybart | 2:50 pm Dec 20, 2004 | ||||
| redirect without confusing spiders |
1 | houseofsecrets | 4:55 pm Dec 13, 2004 |
