Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| sitemap for mobile versions |
2 | sam222 | 2:11 am July 30, 2011 | ||||
| Free tools for creating a sitemap |
11 | Fotiman | 12:13 pm July 27, 2011 | ||||
| Blocking Spiders How to block spiders from crawling your site |
6 | Brian0275 | 9:15 pm July 25, 2011 | ||||
| which part of "disallow" did you not understand? |
9 | lucy24 | 2:46 am July 25, 2011 | ||||
| www, www21, wwww, etc. Displaying Same Content & Indexed By SEs What to do? |
5 | getxb | 6:48 am July 22, 2011 | ||||
| How to manage this in robots.txt disallow a folder but include one page |
10 | pankajj | 1:36 pm July 21, 2011 | ||||
| Disallowed Googlebot-Image bot still spidering |
7 | cyberdyne | 8:47 am July 9, 2011 | ||||
| Archive.org for sites not crawled since 2006 |
8 | Neil_McRae | 6:04 pm July 1, 2011 | ||||
| Googlebot not respecting custom crawl rate |
13 | rowan194 | 3:38 am June 22, 2011 | ||||
| sitemap for images on subdomain |
4 | AlexB77 | 6:50 am June 11, 2011 | ||||
| What's the difference between bing bot and Microsoft? Microsoft IP staying too long...banned for now, what's with that? |
3 | erlandc | 10:21 pm June 4, 2011 | ||||
| Google, Bing and Yahoo agree on Microformats? Was that discussed here: schema.org? |
5 | pontifex | 11:00 pm June 3, 2011 | ||||
| robots.txt wildcard syntax? is this syntax ok? |
5 | maha | 9:39 pm May 31, 2011 | ||||
| Index Page Removal, Serious Problem Please Help robots.txt or whats the issue? non-technical person here |
6 | faust1 | 10:25 am May 29, 2011 | ||||
| Generating a huge sitemap on shared host Creating a sitemap with thousands of URL'son a shared hosting plan. |
1 | BrainDed | 5:54 pm May 23, 2011 | ||||
| robots.txt - disallow any url containing. using robots.txt and dynamic urls for phpbb3 forum |
7 | BrainDed | 10:28 pm May 18, 2011 | ||||
| Why Indexing count Fluctuates? |
2 | ihavegotyou | 11:35 pm May 17, 2011 | ||||
| My SITEMAPS are getting indexed--pls help implementing this tag |
12 | donna130 | 1:16 pm May 14, 2011 | ||||
| the good, the bad and the above the law? |
1 | lucy24 | 10:03 pm May 12, 2011 | ||||
| blocking dynamic parts of forum |
8 | viggen | 6:42 pm May 10, 2011 | ||||
| Sky IP requesting all files alphabetically |
3 | cyberdyne | 3:42 pm May 7, 2011 | ||||
| allowing & disallowing (robots.txt) |
8 | mike2010 | 11:38 pm May 4, 2011 | ||||
| Google, robots, disallow |
3 | cyberdyne | 6:01 pm May 3, 2011 | ||||
| Website not showing in any search engine |
21 | jesstp | 3:20 pm Apr 17, 2011 | ||||
| updating sitemaps - how often? |
5 | iWebTrack | 10:38 am Apr 17, 2011 |
