Welcome to WebmasterWorld Guest from 126.96.36.199
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Regarding question on sitemaps and http to https
||3||Boulos||2:51 am Mar 22, 2018|
|Structured Data Mark Up
Need to know which mark up data for the website
|7||Irfan_Ansari||1:14 pm Feb 20, 2018|
Need to Correct Answer or Solution for My Query
|11||Irfan_Ansari||7:24 am Feb 20, 2018|
|Deleted old articles so now have to update sitemap?
||3||born2run||4:01 am Feb 18, 2018|
|move to https and sitemaps
||10||fatpeter||11:18 am Jan 23, 2018|
|“google sent me”
||5||lucy24||7:27 pm Jan 12, 2018|
|New "Well-Known Uniform Resource Identifiers"
Well-Known Uniform Resource Identifiers are defined in RFC-5785
|5||Andova_Begarin||6:19 pm Dec 8, 2017|
|robots.txt: prevent new bots en masse
configuring robots.txt to black all bots but the ones you want
|6||Andova_Begarin||6:46 am Nov 27, 2017|
|Static site plus blog on root subfolder – separate site maps or not?
||3||jason2004||12:02 pm Nov 6, 2017|
|robots.txt and HTTPS
||8||lucy24||11:50 am Oct 19, 2017|
|Is there a way to block thumbnails in robots.txt?
||7||virtualreality||11:06 pm Sep 14, 2017|
|The Rise & Fall of Robots.txt
Perspective on the Robots Text Standard
|2||keyplyr||4:18 am Sep 8, 2017|
|Excluding image urls from being indexed by robots.
||13||born2run||12:43 am Sep 5, 2017|
|My site gets edited every 4-5 hours, the revisit after metatag value?
||9||born2run||12:42 am Aug 27, 2017|
|Am I going to be punished for NOT having a sitemap?
||5||csdude55||11:48 pm Jun 20, 2017|
|My website inner pages are not crawling and indexed?
||5||Movierola||10:17 am May 18, 2017|
|Strange 404's showing up in GSC
 ( 1 2 )
|40||NickMNS||1:40 pm Apr 24, 2017|
|Sudden drop in traffic from blog on subdomain
figuring out the reason in traffic from blog
|1||Sachet||2:33 pm Mar 17, 2017|
|Site not indexed?
||12||kingbottler||10:11 am Mar 14, 2017|
|Which tags to keep or toss
||6||csdude55||8:25 am Feb 7, 2017|
|Disallow images with suffix
||4||terrybarnes||12:28 pm Jan 13, 2017|
|webmaster SiteLink option
I can't find search appearance > sitelink option
|2||Shihab||12:43 pm Jan 10, 2017|
|Sitemaps File Size up to 50Mb
||4||engine||11:27 pm Dec 12, 2016|
|Blog Tag and Author page blocking with robots.txt
||5||seojustin||6:27 am Dec 3, 2016|
|Robots - Disallow All Except Images, JS & CSS
||2||almo136||11:02 pm Oct 5, 2016|