Forum Moderated by: goodroi
Using a robots.txt is all part of being a good SEO.
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| “google sent me” |
5 | lucy24 | 7:27 pm Jan 12, 2018 | ||||
| New "Well-Known Uniform Resource Identifiers" Well-Known Uniform Resource Identifiers are defined in RFC-5785 |
5 | Andova_Begarin | 6:19 pm Dec 8, 2017 | ||||
| robots.txt: prevent new bots en masse configuring robots.txt to black all bots but the ones you want |
6 | Andova_Begarin | 6:46 am Nov 27, 2017 | ||||
| Static site plus blog on root subfolder – separate site maps or not? |
3 | jason2004 | 12:02 pm Nov 6, 2017 | ||||
| robots.txt and HTTPS |
8 | lucy24 | 11:50 am Oct 19, 2017 | ||||
| Is there a way to block thumbnails in robots.txt? |
7 | virtualreality | 11:06 pm Sep 14, 2017 | ||||
| The Rise & Fall of Robots.txt Perspective on the Robots Text Standard |
2 | keyplyr | 4:18 am Sep 8, 2017 | ||||
| Excluding image urls from being indexed by robots. |
13 | born2run | 12:43 am Sep 5, 2017 | ||||
| My site gets edited every 4-5 hours, the revisit after metatag value? |
9 | born2run | 12:42 am Aug 27, 2017 | ||||
| Am I going to be punished for NOT having a sitemap? |
5 | csdude55 | 11:48 pm Jun 20, 2017 | ||||
| My website inner pages are not crawling and indexed? |
5 | Movierola | 10:17 am May 18, 2017 | ||||
| Strange 404's showing up in GSC [2] ( 1 2 ) |
40 | NickMNS | 1:40 pm Apr 24, 2017 | ||||
| Sudden drop in traffic from blog on subdomain figuring out the reason in traffic from blog |
1 | Sachet | 2:33 pm Mar 17, 2017 | ||||
| Site not indexed? |
12 | kingbottler | 10:11 am Mar 14, 2017 | ||||
| Which tags to keep or toss |
6 | csdude55 | 8:25 am Feb 7, 2017 | ||||
| Disallow images with suffix |
4 | terrybarnes | 12:28 pm Jan 13, 2017 | ||||
| webmaster SiteLink option I can't find search appearance > sitelink option |
2 | Shihab | 12:43 pm Jan 10, 2017 | ||||
| Sitemaps File Size up to 50Mb |
4 | engine | 11:27 pm Dec 12, 2016 | ||||
| Blog Tag and Author page blocking with robots.txt |
5 | seojustin | 6:27 am Dec 3, 2016 | ||||
| Robots - Disallow All Except Images, JS & CSS |
2 | almo136 | 11:02 pm Oct 5, 2016 | ||||
| Please explain what I do? |
6 | williamgeff | 6:59 am Sep 19, 2016 | ||||
| Is it normal for a browser to request filelist.xml and editdata.mso? what or why is filelist.xlm and editdata.mso being requested? |
1 | SumGuy | 3:01 am Sep 8, 2016 | ||||
| no indexing only certain pages in wordpress |
2 | romerome | 7:26 pm Sep 5, 2016 | ||||
| bizarre robots.txt must be an old WebmasterWorld suggestion for robots.txt |
3 | AussieWebmaster | 11:48 pm Jul 20, 2016 | ||||
| Which Wordpress Plugin is Best for Sitemap |
3 | rg450348 | 10:24 am Jul 18, 2016 |
