Welcome to WebmasterWorld Guest from 18.104.22.168
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|The best way to add uncrawlable links?
|6||jonny0000||12:15 pm Mar 18, 2010|
|After using robots.txt none of the bots are crawling my site! Help
robots.txt is not working properly
|10||shamims||12:11 pm Mar 18, 2010|
|nofollow tag or prevent robots from visiting /admin page?
nofollow and robot.txt
|3||parliamenttutors||7:17 pm Mar 16, 2010|
|SEO Friendly Collapsable Content
There are several SEO friendly ways of displaying collapsable content. Wond
|7||jonny0000||3:23 pm Mar 16, 2010|
|Do these robot.txt's look ok?
||4||Boulder90||1:45 pm Mar 16, 2010|
|Persistent Crawl Errors
||2||GameTestLab||2:28 pm Mar 15, 2010|
|Google cached "disallow all" robots.txt and sitemap re-submit failed
||2||frankruijiang||1:55 pm Mar 5, 2010|
|Moved: question of a newbie - Checking Backlinks
Post was moved here: http://www.webmasterworld.com/link_development/4088876.htm
|1||System||5:45 pm Feb 27, 2010|
|Kilobytes downloaded per day vs Pages crawled per day
Kilobytes downloaded per day vs Pages crawled per day
|4||cocorocks||12:19 pm Feb 22, 2010|
|Robots.txt google webmasters blogger blogspot help
||2||Jcblue||12:17 pm Feb 22, 2010|
|Google site map issues when URL contains %3A
Google is not properly following site map URLs containing a %3A
|9||KenB||10:36 pm Feb 21, 2010|
|Constantly falling indexed pages and organic traffic in Google
Constantly falling indexed pages and organic traffic in Google
|2||cocorocks||4:34 pm Feb 16, 2010|
|Robots.txt and dynamic pages
||3||seasalt||8:03 am Feb 14, 2010|
|Do I need to resubmit a new sitemap for new content?
||8||BitShift||7:47 am Feb 14, 2010|
|SWF content and robots.txt
swf, robots.txt, google index, removal request, sandbox
|4||rob1m1982||8:11 pm Feb 12, 2010|
Should I block it?
|4||maximillianos||3:48 am Feb 10, 2010|
|Tool to check if multiple URLs are indexed?
||5||DiscoStu||9:25 am Feb 4, 2010|
|3 years on: robots-nocontent, noindex, and google ad section start
Identifying core or non-core content on a page
|1||brycen||10:24 pm Feb 3, 2010|
|Sitemap.xml : Hide from Public view?
||6||donna130||2:13 am Feb 3, 2010|
|robots.txt: Disallow: /links.htm
I want to disallow "links.htm"
|3||leogang||10:34 am Feb 2, 2010|
|Looking for good wordpress robots.txt
||2||Boulder90||4:51 pm Feb 1, 2010|
|Robots.txt not working
||3||samit||5:38 am Jan 28, 2010|
|Got a mistake on Google Webmaster Tools during removing URL
This problem was occured when i wanted to remove my old cache pages
|3||shamims||1:16 pm Jan 16, 2010|
|ROBOTS.txt---Yes to Google, but No to Public view
||4||donna130||11:24 am Jan 11, 2010|
|Proxy Hijacking or 302 Redirect is It Completely Fixed By G?
||2||herculano||8:16 pm Jan 10, 2010|