Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Site Pages are Blocked by Robots.txt |
4 | bilalseo | 10:03 am May 6, 2008 | ||||
| robots.txt and IP addresses |
11 | Maciej_Ziemczonek | 10:11 pm May 2, 2008 | ||||
| Is this valid? wildcard in robots Disallow: /*/edit.html |
3 | rudyten | 1:47 pm Apr 20, 2008 | ||||
| Will spiders follow links on URL's blocked by robots.txt |
3 | deanw | 6:34 pm Apr 16, 2008 | ||||
| Security Challenges |
4 | pageoneresults | 3:49 pm Apr 2, 2008 | ||||
| Dumbest question eva: Do all major SEs support the Google guidelines? I am testing mine on G's WebmasterCentral |
2 | walkman | 6:42 pm Mar 26, 2008 | ||||
| Is there any way to block everyone except. for Google, Yahoo etc? |
2 | mmiller | 9:41 am Mar 21, 2008 | ||||
| need to block. and don't know - pls help need help with blocking - Malware Ref |
4 | Reilly | 7:55 pm Mar 17, 2008 | ||||
| How to block original + localized pages? robots.txt localized |
1 | vanguard2000 | 6:19 pm Mar 17, 2008 | ||||
| How long before bots re-visit after they were told they are not allowed? |
3 | boromir27 | 3:32 am Mar 17, 2008 | ||||
| Inktomi User-Agent blocked Inktomi blocked |
6 | cyberdyne | 8:36 am Mar 14, 2008 | ||||
| robots.txt - Disallow/Allow |
7 | eyalkattan | 1:14 am Mar 13, 2008 | ||||
| robots and redirecting i am wondering how a robot would act on a page with a redirect |
2 | melanger | 8:56 pm Mar 8, 2008 | ||||
| Unusual log entries |
5 | cyberdyne | 6:17 pm Mar 8, 2008 | ||||
| Redirect robot.txt from http to https how to redirect properly. |
8 | clyde4210 | 2:59 pm Mar 6, 2008 | ||||
| Uh oh, it's happening again robots.txt blocked out |
2 | wesg | 9:58 pm Feb 28, 2008 | ||||
| new badbot: EnaBot/1.0 |
7 | Achernar | 8:31 am Feb 28, 2008 | ||||
| Need help blocking search results page Lots of keyword search pages indexed, some with no content |
4 | ecmedia | 2:28 pm Feb 26, 2008 | ||||
| disallow ALL bots.and then allow the good ones? |
5 | tokey666 | 8:40 pm Feb 25, 2008 | ||||
| Excluding a stub page with robots.txt Dealing with trailing forward slash |
4 | bouncybunny | 12:41 am Feb 24, 2008 | ||||
| What is wrong with my Robots.txt file? I'd appreciate help with my Robots.txt file |
7 | Morthan | 3:07 am Feb 22, 2008 | ||||
| Help with blocking URLS |
5 | jaffstar | 7:17 am Feb 18, 2008 | ||||
| 301 Redirect and Search Ranking 301 Redirect and Search Ranking |
4 | bilalseo | 6:22 am Feb 18, 2008 | ||||
| No life without robots.txt file This may be old hat to you but it knocked me for six |
18 | folkranger | 7:17 pm Feb 15, 2008 | ||||
| Robots.txt and iframes How to disallow iframes using robots.txt? |
6 | foxtunes | 11:00 pm Feb 14, 2008 |
