Welcome to WebmasterWorld Guest from 54.82.29.141

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
“google sent me”
5 lucy24 7:27 pm Jan 12, 2018
New "Well-Known Uniform Resource Identifiers"
Well-Known Uniform Resource Identifiers are defined in RFC-5785
5 Andova_Begarin 6:19 pm Dec 8, 2017
move to https and sitemaps
9 fatpeter 2:03 pm Dec 6, 2017
robots.txt: prevent new bots en masse
configuring robots.txt to black all bots but the ones you want
6 Andova_Begarin 6:46 am Nov 27, 2017
Static site plus blog on root subfolder – separate site maps or not?
3 jason2004 12:02 pm Nov 6, 2017
robots.txt and HTTPS
8 lucy24 11:50 am Oct 19, 2017
Is there a way to block thumbnails in robots.txt?
7 virtualreality 11:06 pm Sep 14, 2017
The Rise & Fall of Robots.txt
Perspective on the Robots Text Standard
2 keyplyr 4:18 am Sep 8, 2017
Excluding image urls from being indexed by robots.
13 born2run 12:43 am Sep 5, 2017
My site gets edited every 4-5 hours, the revisit after metatag value?
9 born2run 12:42 am Aug 27, 2017
Am I going to be punished for NOT having a sitemap?
5 csdude55 11:48 pm Jun 20, 2017
My website inner pages are not crawling and indexed?
5 Movierola 10:17 am May 18, 2017
Strange 404's showing up in GSC
[2] ( 1 2 )
40 NickMNS 1:40 pm Apr 24, 2017
Sudden drop in traffic from blog on subdomain
figuring out the reason in traffic from blog
1 Sachet 2:33 pm Mar 17, 2017
Site not indexed?
12 kingbottler 10:11 am Mar 14, 2017
Which tags to keep or toss
6 csdude55 8:25 am Feb 7, 2017
Disallow images with suffix
4 terrybarnes 12:28 pm Jan 13, 2017
webmaster SiteLink option
I can't find search appearance > sitelink option
2 Shihab 12:43 pm Jan 10, 2017
Sitemaps File Size up to 50Mb
4 engine 11:27 pm Dec 12, 2016
Blog Tag and Author page blocking with robots.txt
5 seojustin 6:27 am Dec 3, 2016
Robots - Disallow All Except Images, JS & CSS
2 almo136 11:02 pm Oct 5, 2016
Please explain what I do?
6 williamgeff 6:59 am Sep 19, 2016
Is it normal for a browser to request filelist.xml and editdata.mso?
what or why is filelist.xlm and editdata.mso being requested?
1 SumGuy 3:01 am Sep 8, 2016
no indexing only certain pages in wordpress
2 romerome 7:26 pm Sep 5, 2016
bizarre robots.txt
must be an old WebmasterWorld suggestion for robots.txt
3 AussieWebmaster 11:48 pm Jul 20, 2016

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members