Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| WebmasterWorldForumBot? what is this? |
2 | Trisha | 10:33 pm Feb 10, 2004 | ||||
| How to stop robots on .php files |
3 | djtaverner | 6:40 pm Feb 10, 2004 | ||||
| What in the world is a robot.txt? I need this to be stated simple for me |
5 | Claudia_T | 6:35 pm Feb 10, 2004 | ||||
| Help! I need to disallow a specific query string |
1 | Hissingsid | 11:26 am Feb 10, 2004 | ||||
| Please verify |
2 | mikemcs | 4:25 am Feb 10, 2004 | ||||
| Googlebot chasing .php tracking script is robots.txt enough? |
2 | Edouard_H | 12:51 am Feb 9, 2004 | ||||
| 301 redirects & robots.txt What is the implication of a 301 redirect on robots.txt |
2 | Jerry_Pannu | 10:17 pm Feb 7, 2004 | ||||
| Testing robots.txt Why won't my site get spidered |
2 | DavidAtWork | 5:56 pm Feb 5, 2004 | ||||
| I've been looking at the WW robots.txt ... ...but why are these spiders disallowed? |
7 | elgumbo | 1:32 pm Feb 4, 2004 | ||||
| GoogleBot Will using robots exclusion and meta exclusion hurt me? |
1 | mercury | 9:12 pm Feb 3, 2004 | ||||
| Spiders that ignore or skip robots.txt Can bad spiders be identified? |
4 | DavidAtWork | 2:05 pm Feb 3, 2004 | ||||
| Robots Meta and .txt Where? |
4 | admin_amoroustoys | 1:34 pm Feb 3, 2004 | ||||
| robots.txt used in court case |
2 | transactiongeek | 1:27 pm Feb 3, 2004 | ||||
| what is the "KAIST AITrc Crawler" bot? it ignored my "robots.txt" file and got caught in my bot trap |
4 | stapel | 12:01 pm Feb 3, 2004 | ||||
| Is it illegal to ignore a robots.txt? Could I sue google? |
2 | transactiongeek | 6:44 am Feb 3, 2004 | ||||
| robots metatag I have noindex,follow but the page is indexed |
2 | nevetS | 4:41 am Feb 3, 2004 | ||||
| save and upload robots.txt |
4 | helenp | 8:27 pm Feb 2, 2004 | ||||
| Is this robots.txt blocking anybody |
8 | nalavanje | 2:20 pm Feb 2, 2004 | ||||
| Is this enough |
5 | dougie | 11:20 am Feb 2, 2004 | ||||
| How Do I Code My Robots.txt File To Disallow Subdomians? Is it Disallow: /apple.domain.com or Disallow: /apple/ |
2 | Christyl_Stevens | 9:34 am Feb 2, 2004 | ||||
| asterias/2.0 Looks for streaming media files... |
6 | pendanticist | 11:43 pm Feb 1, 2004 | ||||
| will all robots crawl the sites without robot.txt? robot crawl, robot.txt |
2 | johnlim | 3:08 am Feb 1, 2004 | ||||
| What will Inktomi do in this scenario? |
18 | flobaby | 6:02 am Jan 31, 2004 | ||||
| Google spidering past secure server Robots.txt file to boot the bot? |
2 | xbase234 | 12:52 am Jan 31, 2004 | ||||
| Logs keeping showing 404 error I have a robots file and it looks ok |
3 | eesnard | 12:03 am Jan 30, 2004 |
