Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| question about robots.txt, am I banning a directory by accident |
17 | arbitrary | 3:32 am Jan 15, 2006 | ||||
| Quick 'Sanity Check' |
3 | markd | 10:20 am Jan 11, 2006 | ||||
| directory |
4 | sylvieg | 4:38 am Jan 11, 2006 | ||||
| Removing pages from Google's cache Is it best to allow a "robots" "noarchive" crawl first? |
1 | Phil_Payne | 2:05 am Jan 10, 2006 | ||||
| Disallow ALL folders? but allow top level html |
6 | Reno | 9:14 pm Jan 9, 2006 | ||||
| How to disallow widgets from being indexed Order of User-agents |
2 | albatros | 8:10 pm Jan 9, 2006 | ||||
| Empty line after User-agent: Could this do any harm? |
1 | taps | 5:31 pm Jan 7, 2006 | ||||
| Robot Validator Problem |
7 | bunky6767 | 1:13 pm Jan 6, 2006 | ||||
| Blocking all URLS to a website using the robots.txt file? |
2 | scubajared | 10:05 am Jan 6, 2006 | ||||
| Turing keys |
2 | Ejap | 5:57 pm Jan 5, 2006 | ||||
| ABC of robots.txt Now that will be the best tutorial! |
8 | Monzilla | 2:19 pm Jan 4, 2006 | ||||
| Define allowable spider times MSN is going nutz![2] ( 1 2 ) |
31 | Reno | 1:44 pm Jan 4, 2006 | ||||
| New empty site beeing spidered |
2 | sren | 3:08 am Dec 29, 2005 | ||||
| remove PHP from Google.com remove PHP from Google.com |
2 | maimax | 7:32 am Dec 28, 2005 | ||||
| paging google! robots.txt being ignored! |
14 | elvey | 7:25 am Dec 27, 2005 | ||||
| How to block robots spidering certain pages? |
4 | zoltan | 6:42 pm Dec 23, 2005 | ||||
| Correct Syntax for Blogs in robots.txt |
3 | vincevincevince | 4:27 pm Dec 22, 2005 | ||||
| Blocking domain specific links |
3 | seasalt | 11:33 pm Dec 21, 2005 | ||||
| Google appears to be ignoring robots file No change after three months |
5 | Lokutus | 9:50 pm Dec 16, 2005 | ||||
| parameter page to be banned from crawling directory.asp? |
4 | idolw | 1:18 pm Dec 15, 2005 | ||||
| About the robots tag for Googlebot Is it necessary? |
2 | a1call | 8:57 pm Dec 14, 2005 | ||||
| Remove part of your website Should I use robots.txt or 301 redirects? |
1 | twist | 7:27 pm Dec 13, 2005 | ||||
| Robots.txt and the Google sandbox Does de-indexing put one back into the sandbox? |
2 | Rollo | 7:05 pm Dec 13, 2005 | ||||
| Should I use the "?" or not. |
2 | kgormat | 3:52 am Dec 12, 2005 | ||||
| robots.txt and the law do they intersect |
2 | Powdork | 10:19 am Dec 11, 2005 |
