Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Question about robots meta tag, as well as robots.txt.... |
3 | xsirrionx | 11:48 am May 20, 2004 | ||||
| How to validate robots.txt on "https" site? Please help me validate my "https" site |
2 | CodeCrunch | 12:29 am May 20, 2004 | ||||
| Wild chars *.* allowed in "Disallow" of Robots.txt? or anything else like *.txt, etc. |
2 | CodeCrunch | 12:26 am May 20, 2004 | ||||
| robots.txt restrictions conflictual with site's Terms Which of them should prevail? |
4 | cristiscu | 6:52 am May 19, 2004 | ||||
| Comments at the end of a line bad style, why? |
3 | zachariah | 6:52 am May 18, 2004 | ||||
| Spider Testing |
6 | cabowabo | 3:44 am May 13, 2004 | ||||
| Meta Tags & Robots.txt |
4 | tcustom | 1:57 am May 13, 2004 | ||||
| Why is Google still indexing my SWF files? |
2 | ichthyous | 6:10 am May 10, 2004 | ||||
| Deny or Not to Deny /images Denying /images folder |
9 | quebrada | 12:19 am May 6, 2004 | ||||
| proper way of changing site |
2 | jmdb71 | 5:11 pm May 1, 2004 | ||||
| How to get spider to visit parts of site more frequently Is this possible? |
5 | Pimpernel | 10:35 am Apr 30, 2004 | ||||
| robots.txt missing? |
2 | zyshen | 5:19 pm Apr 27, 2004 | ||||
| Google ignoring robots.txt I'm seeing lots of my 'Disallowed' pages in their index |
4 | Amygdala | 4:54 am Apr 24, 2004 | ||||
| Strange bot keep on visiting my site and don't obey my robots.txt file What to do? |
2 | Compworld | 8:01 pm Apr 22, 2004 | ||||
| revisit after so many days |
2 | tommygunner | 3:21 am Apr 21, 2004 | ||||
| index.php vs index.html Will This Work i Wonder |
2 | yummycarp | 11:28 pm Apr 20, 2004 | ||||
| Banning a script using robots.txt how do do this |
4 | Imaster | 9:51 pm Apr 20, 2004 | ||||
| Robots.txt - Why? |
3 | warlordbb | 2:08 pm Apr 20, 2004 | ||||
| Can IMDB do this? charge per unwanted request |
3 | jrap | 10:19 pm Apr 19, 2004 | ||||
| Robots.txt exclusion Google keeps crawling page |
2 | anxvariety | 2:27 pm Apr 19, 2004 | ||||
| Froogle wants my images! |
3 | seadog | 5:59 am Apr 14, 2004 | ||||
| META Robots no index tag |
2 | BMach | 2:36 am Apr 14, 2004 | ||||
| Using wild card in robots Can you add * to ip? |
2 | walrus | 5:37 pm Apr 13, 2004 | ||||
| How to ban inktomisearch.com's bots? I have been hit hard with so many requests by this. |
3 | itisgene | 5:24 pm Apr 11, 2004 | ||||
| Validation Error |
4 | decdim | 12:12 am Apr 11, 2004 |
