Welcome to WebmasterWorld Guest from 220.127.116.11
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|sitemap for images on subdomain
||4||AlexB77||6:50 am Jun 11, 2011|
|What's the difference between bing bot and Microsoft?
Microsoft IP staying too long...banned for now, what's with that?
|3||erlandc||10:21 pm Jun 4, 2011|
|Google, Bing and Yahoo agree on Microformats?
Was that discussed here: schema.org?
|5||pontifex||11:00 pm Jun 3, 2011|
|robots.txt wildcard syntax?
is this syntax ok?
|5||maha||9:39 pm May 31, 2011|
|Index Page Removal, Serious Problem Please Help
robots.txt or whats the issue? non-technical person here
|6||faust1||10:25 am May 29, 2011|
|Generating a huge sitemap on shared host
Creating a sitemap with thousands of URL'son a shared hosting plan.
|1||BrainDed||5:54 pm May 23, 2011|
|robots.txt - disallow any url containing.
using robots.txt and dynamic urls for phpbb3 forum
|7||BrainDed||10:28 pm May 18, 2011|
|Why Indexing count Fluctuates?
||2||ihavegotyou||11:35 pm May 17, 2011|
|My SITEMAPS are getting indexed--pls help implementing this tag
||12||donna130||1:16 pm May 14, 2011|
|the good, the bad and the above the law?
||1||lucy24||10:03 pm May 12, 2011|
|blocking dynamic parts of forum
||8||viggen||6:42 pm May 10, 2011|
|Sky IP requesting all files alphabetically
||3||cyberdyne||3:42 pm May 7, 2011|
|allowing & disallowing (robots.txt)
||8||mike2010||11:38 pm May 4, 2011|
|Google, robots, disallow
||3||cyberdyne||6:01 pm May 3, 2011|
|Website not showing in any search engine
||21||jesstp||3:20 pm Apr 17, 2011|
|updating sitemaps - how often?
||5||iWebTrack||10:38 am Apr 17, 2011|
|how many pages should be added in xml site map
||2||getxmlsitemap||10:25 am Apr 17, 2011|
|Redireted URls showing up in Robots.txt Errors
||3||raj1094||8:58 am Apr 13, 2011|
|name of the website in title tag
||4||iWebTrack||7:36 am Apr 12, 2011|
|Meta.Txt is it real?
Is Meta.Txt an accepted protocol?
|2||iPullRank||10:47 am Apr 7, 2011|
|Help: htaccess rewrite rule
||14||bramley||7:27 am Apr 3, 2011|
|Help: robots disallow pages with various query-strings
||10||bramley||12:24 am Apr 1, 2011|
|Sitemap.xml giving errors for & and &
||4||darshanaasodekar||8:15 am Mar 26, 2011|
change to high performing robots.txt
|3||The_Nountaineer||7:17 pm Mar 21, 2011|
|Crawler software for really big sites
Do you know any that can crawl up to 1 million pages?
|3||Josuah||4:33 pm Mar 21, 2011|