Welcome to WebmasterWorld Guest from 54.226.25.231

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Robots.txt Was Inaccessible
Crawl Postponed Because Robots.txt Was Inaccessible
7 poojabhatt 4:21 pm Sep 16, 2012
Moved: In page analytics
Post was moved here: http://www.webmasterworld.com/analytics/4491502.htm
1 System 11:25 am Sep 4, 2012
Google Index robots.txt
google describes the result for the pages blcocked by robots.txt
9 webseosolution 5:41 am Aug 30, 2012
Google appears to be ignoring 301'd URLs
Googlebot indexing site by IP, even though server is set up to 301
13 rowan194 6:50 pm Aug 28, 2012
Blocking sub-domain in robots.txt?
5 shaunm 10:24 am Aug 23, 2012
Apache: execute "global" php script when robots.txt requested
Is there a way to make Apache run a script for a load of a particular file?
10 rowan194 7:53 pm Aug 20, 2012
How do I block a sub-domain?
How do I block a dub-domain if the sub-domain is a virtual directory?
6 shaunm 10:38 am Aug 17, 2012
Different search result for searching single and plural word
2 bubble 2:43 pm Aug 13, 2012
Googlebot possibly ignoring robots.txt
23 Andem 6:27 am Jul 29, 2012
Noindex vs Disallow within a Sitemap
What is the use of using Noindex within a Sitemap?
9 shaunm 5:28 am Jul 27, 2012
google duplicate title issue
7 jon22 10:02 am Jul 6, 2012
ccTLDs and Sitemaps
Can I have one XML sitemap for all my ccTLDs?
3 shaunm 9:35 am Jun 25, 2012
HTTP 320 URLs, robots.txt and Google
I need to remove indexed HTTP 302 Redirect URLs from Google index.
5 nabucosound 12:38 am Jun 16, 2012
Robots Follow Meta Tag
Will nofollow attribute override robots follow meta tag.
4 Hoegaarden 1:20 am Jun 9, 2012
block ? with robots.txt or htaccess
3 zeus 10:52 pm Jun 8, 2012
Sitemap Problem: Google shows sitemap URL in results instead of post U
2 Lemans 10:44 pm Jun 8, 2012
Robots.txt and regular expressions
Can they use them?
10 jqwan 7:48 pm Jun 4, 2012
Mozilla/4.* (Compatible
to block or not?
13 Seedy 9:25 pm May 21, 2012
Meta tag Rel="canonical" & double content
How to use this?
15 textex 1:59 am May 20, 2012
Robots.txt for canonical preference
having dup content issues
5 textex 9:23 am May 19, 2012
Title and Description Meta Tags
4 ultrafade 12:08 am May 17, 2012
180.x.x.x
3 Seedy 7:47 am May 15, 2012
Crawl time
crawlers
1 satty 12:34 pm Apr 28, 2012
No Follow on Redirected Pages
Will 'no follow' on prevent transfer of SEO juice on redirected pages?
5 ablu272 1:40 pm Apr 16, 2012
Sitemap processing changed?
4 HitProf 9:18 pm Apr 4, 2012

Featured Threads

Hot Threads This Week

Hot Threads This Month