Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Crawl Rate: How do I know when you request slow versus fast Does a long maximum time spent crawling a page have adverse impact on targe |
2 | 11364guy | 7:50 pm Oct 6, 2008 | ||||
| Non-white list blocking With a sting |
4 | phred | 12:51 pm Sept 30, 2008 | ||||
| How to attract Yahoo! Slurp Spider |
2 | ouarsenis | 1:28 pm Sept 25, 2008 | ||||
| My website all pages are in cache but home page are not caching home page is not caching , what should i do for it? |
3 | pervezalam | 6:43 pm Sept 24, 2008 | ||||
| robots , duplicate content and google |
9 | fsmobilez | 8:07 pm Sept 23, 2008 | ||||
| Robots.txt Ive never used robots.txt |
4 | srobinson | 3:21 am Sept 23, 2008 | ||||
| Understanding the structure of robots.txt subdirectories |
10 | alahamdan | 12:00 am Sept 5, 2008 | ||||
| Baiduspider - how do I keep it out. having some difficulty blocking this spider |
3 | ChicagoFan67 | 1:44 pm Sept 4, 2008 | ||||
| restrict special duplicate pages |
4 | sixeleven | 11:58 pm Sept 3, 2008 | ||||
| How to set Slow GoogleBot Crawl Speed I need to fix Google to crawl my site as very slow or specifice times, how? |
7 | Senthil | 7:54 am Aug 31, 2008 | ||||
| Errors of robots.txt unreachable in google webmaster sitemap Please anyone help me |
1 | pervezalam | 9:35 am Aug 29, 2008 | ||||
| What is Robot.txt? What is robot.txt, |
8 | deepk | 7:16 am Aug 21, 2008 | ||||
| disallow & allow |
3 | doumiao | 6:14 am Aug 15, 2008 | ||||
| Regular Expressions, robots.txt, and robotstxt.org |
4 | Funtick | 7:24 pm Aug 14, 2008 | ||||
| How to disallow https urls How to disallow https urls |
6 | spiritualseo | 8:39 pm Aug 12, 2008 | ||||
| URLs restricted by robots.txt How to remove that restricton or reinclude URL |
2 | Raheel | 2:28 pm Aug 9, 2008 | ||||
| google search box |
1 | sixeleven | 3:10 pm Aug 8, 2008 | ||||
| Google has indexed my sitemap.xml sitemap.xml appears as first result in serps for my domain |
2 | wrightee | 1:20 pm Aug 4, 2008 | ||||
| How to disallow subfolders |
3 | bonagiri | 10:03 am July 29, 2008 | ||||
| My website ranking dropped! after i create a wrong robotx.txt... |
9 | virtualreality | 1:07 am July 29, 2008 | ||||
| Blocking parts of forum but not others |
11 | AnonyMouse | 8:24 pm July 28, 2008 | ||||
| i should disallow ,second pages? |
3 | sixeleven | 2:37 pm July 25, 2008 | ||||
| How to disallow search results How to disallow search results |
6 | spiritualseo | 2:01 pm July 17, 2008 | ||||
| Discrepancy between Webmaster Tools robots.txt analysis and reality |
6 | fishfinger | 10:05 am July 14, 2008 | ||||
| googlebot sitemap question thinking about submitting sitemap, but not decided yet |
5 | cirlong | 2:38 pm July 13, 2008 |
