Welcome to WebmasterWorld Guest from 54.196.224.166

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
After using robots.txt none of the bots are crawling my site! Help
robots.txt is not working properly
10 shamims 12:11 pm Mar 18, 2010
nofollow tag or prevent robots from visiting /admin page?
nofollow and robot.txt
3 parliamenttutors 7:17 pm Mar 16, 2010
SEO Friendly Collapsable Content
There are several SEO friendly ways of displaying collapsable content. Wond
7 jonny0000 3:23 pm Mar 16, 2010
Do these robot.txt's look ok?
4 Boulder90 1:45 pm Mar 16, 2010
Persistent Crawl Errors
2 GameTestLab 2:28 pm Mar 15, 2010
Google cached "disallow all" robots.txt and sitemap re-submit failed
2 frankruijiang 1:55 pm Mar 5, 2010
Moved: question of a newbie - Checking Backlinks
Post was moved here: http://www.webmasterworld.com/link_development/4088876.htm
1 System 5:45 pm Feb 27, 2010
Kilobytes downloaded per day vs Pages crawled per day
Kilobytes downloaded per day vs Pages crawled per day
4 cocorocks 12:19 pm Feb 22, 2010
Robots.txt google webmasters blogger blogspot help
2 Jcblue 12:17 pm Feb 22, 2010
Google site map issues when URL contains %3A
Google is not properly following site map URLs containing a %3A
9 KenB 10:36 pm Feb 21, 2010
Constantly falling indexed pages and organic traffic in Google
Constantly falling indexed pages and organic traffic in Google
2 cocorocks 4:34 pm Feb 16, 2010
Robots.txt and dynamic pages
3 seasalt 8:03 am Feb 14, 2010
Do I need to resubmit a new sitemap for new content?
8 BitShift 7:47 am Feb 14, 2010
SWF content and robots.txt
swf, robots.txt, google index, removal request, sandbox
4 rob1m1982 8:11 pm Feb 12, 2010
Yandex bot
Should I block it?
4 maximillianos 3:48 am Feb 10, 2010
Tool to check if multiple URLs are indexed?
5 DiscoStu 9:25 am Feb 4, 2010
3 years on: robots-nocontent, noindex, and google ad section start
Identifying core or non-core content on a page
1 brycen 10:24 pm Feb 3, 2010
Sitemap.xml : Hide from Public view?
6 donna130 2:13 am Feb 3, 2010
robots.txt: Disallow: /links.htm
I want to disallow "links.htm"
3 leogang 10:34 am Feb 2, 2010
Looking for good wordpress robots.txt
2 Boulder90 4:51 pm Feb 1, 2010
Robots.txt not working
3 samit 5:38 am Jan 28, 2010
Got a mistake on Google Webmaster Tools during removing URL
This problem was occured when i wanted to remove my old cache pages
3 shamims 1:16 pm Jan 16, 2010
ROBOTS.txt---Yes to Google, but No to Public view
4 donna130 11:24 am Jan 11, 2010
Proxy Hijacking or 302 Redirect is It Completely Fixed By G?
2 herculano 8:16 pm Jan 10, 2010
Problem with Google News Sitemap File
4 anand84 4:51 pm Jan 7, 2010

Featured Threads

Hot Threads This Week

Hot Threads This Month