Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Add Sitemap To robots.txt For Autodiscovery Ask.com, Google, Microsoft Live Search and Yahoo! [2] ( 1 2 ) |
32 | OldWolf | 7:56 am May 4, 2007 | ||||
| How is alexa getting past my stoppers stop alexa index |
1 | Michel_Samuel | 3:31 pm May 2, 2007 | ||||
| how to disallow all robots except Googlebot, Yahoo Slurp and MSNBot? |
8 | d6rth7ader | 5:48 am May 2, 2007 | ||||
| Htaccess |
3 | harish_s | 2:17 pm May 1, 2007 | ||||
| Limiting the depth of Google's crawl? Can robots.txt be used to limit depth |
6 | daveozzz | 7:21 pm Apr 27, 2007 | ||||
| Disallow: / How did googlebot index my web server dir structure? |
3 | thinkweb | 5:24 am Apr 26, 2007 | ||||
| URLs restricted by robots.txt (41) AGH! I'm a bit stumped about what's best to do re my robots.txt |
8 | wrafter | 8:38 pm Apr 25, 2007 | ||||
| Is a robots.txt file necessary? if not trying to block bots? |
3 | Jakes_Redding | 5:52 am Apr 20, 2007 | ||||
| Googlebot ignoring robots.txt? Or is my robots.txt file wrong... |
3 | designaweb | 9:28 am Apr 17, 2007 | ||||
| Blocking Google Only From certain sub domains |
5 | Moncao | 7:25 pm Apr 13, 2007 | ||||
| unknown robot robot |
4 | kool002 | 1:59 pm Apr 11, 2007 | ||||
| Not being found by the search engines noindex,nofollow |
4 | gtate | 6:24 pm Apr 6, 2007 | ||||
| Efficient way to block multiple bots? |
11 | JAB_Creations | 2:33 pm Apr 3, 2007 | ||||
| Blocking Google I need to block google from my index page |
4 | loner | 4:25 pm Apr 2, 2007 | ||||
| help needed |
1 | hugh | 4:03 pm Apr 2, 2007 | ||||
| What's the Proper Way to Block Sub Folders Looking for the best way to block 2nd level sub folders |
5 | graywolf | 7:43 pm Mar 28, 2007 | ||||
| ROBOT and PR rank? how does the robot effect your pr rank? |
7 | moppy56 | 1:48 pm Mar 28, 2007 | ||||
| Blocking Partial Contents of Folder Without Using Robots Tag 10 files (7 want indexed, 3 want blocked) |
5 | SandySEO | 8:49 pm Mar 27, 2007 | ||||
| Another bad bot to ban this one is using 302 redirects .. bad bad |
1 | Bewenched | 3:43 am Mar 25, 2007 | ||||
| excluding a subdomain Using the robots.txt file to exclude a subdomain |
10 | joebray | 9:47 am Mar 23, 2007 | ||||
| G. ignores robots.txt at other ports, gives false page count Google shows in its index pages that it never saw, even prohibited |
1 | OnetSzukaj | 2:58 pm Mar 22, 2007 | ||||
| How to use Robots.txt as an INCLUSION method It's exclusion only if you use it that way |
1 | incrediBILL | 12:32 am Mar 22, 2007 | ||||
| TripAdvisorBot Anyone seen this? |
3 | bobbarnes | 10:28 pm Mar 16, 2007 | ||||
| problem with msnbot and wildcard matching |
2 | Achernar | 1:36 pm Mar 16, 2007 | ||||
| How Can I Make Bots Visit My Web Site? |
3 | emadido | 2:34 pm Mar 15, 2007 |
