Welcome to WebmasterWorld Guest from 126.96.36.199
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Filenames have spaces - How to reflect in robots.txt
Website has filenames that contain spaces. What to do?
|4||SanDiego_Art||2:54 am Mar 16, 2005|
|restriction for a certain country
is it possible to do that?
|3||lucky||8:06 pm Mar 15, 2005|
|need help blocking a site
not sure if this is possible
|11||willeffects||5:23 am Mar 15, 2005|
||1||driwashsolutions||1:59 am Mar 15, 2005|
Understanding my hits log
|5||cardpro||8:59 am Mar 14, 2005|
|i see the bigger site have the bots not crawling them
i see the bigger site have the bots not crawling them
|2||indodpendo||7:12 pm Mar 13, 2005|
|im using this sites robits.txt file but my main page gets psa's
im using this sites robits.txt file but my main page gets psa's
|2||indodpendo||7:10 pm Mar 13, 2005|
|3||dj_lysuc||12:30 pm Mar 13, 2005|
|is it necessary for dynamic pages
|3||qais77||2:38 pm Mar 12, 2005|
|Only Allowing Googlebot, Msnbot and Slurp
is it possible?
|4||globay||4:28 pm Mar 11, 2005|
|disallow all from subdirectory
||11||Driven||1:26 pm Mar 10, 2005|
|Robots.txt will it be ignored
||1||azzbacwordz||6:39 pm Mar 9, 2005|
I need a robots.txt
|4||indodpendo||1:35 am Mar 9, 2005|
|simple question concerning adsense
||1||esllou||10:26 am Mar 7, 2005|
|Need help with robot.txt for .php forums
robot.txt, .php, forums,
|2||Freakindj||2:44 pm Mar 5, 2005|
|Bot blocking versus IP throttling for bad bots
Dealing with rippers and their ilk
|1||Webwork||4:10 am Mar 3, 2005|
|Does this cancle out the robots.txt
Does this cancle out the robots.txt
|3||indodpendo||2:16 pm Mar 1, 2005|
robot.txt, subdirectories, protected directories
|1||Ansuz||1:47 pm Feb 28, 2005|
|Robot terms defined
The differences and similarities between nofollow, noindex and robots.txt
|1||u_to_the_sixth||7:29 am Feb 26, 2005|
|Maximum size for robots.txt
Does any one know what it is?
|6||blend27||9:40 pm Feb 25, 2005|
|4||indodpendo||8:59 pm Feb 25, 2005|
|How can I get my whole site out of search engines and Google cache?
||2||mig1234||5:19 pm Feb 23, 2005|
|Banning Affiliate Links from being Indexed
||1||bears5122||11:57 pm Feb 22, 2005|
|Advanced robots.txt question
About blocking subdomain on different server.
|4||Buddha||12:57 am Feb 21, 2005|
|robots.txt wildcard matching
How wild is it?
|8||kwasher||12:51 am Feb 21, 2005|