Welcome to WebmasterWorld Guest from 188.8.131.52
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|paging google! robots.txt being ignored!
||14||elvey||7:25 am Dec 27, 2005|
|How to block robots spidering certain pages?
||4||zoltan||6:42 pm Dec 23, 2005|
|Correct Syntax for Blogs in robots.txt
||3||vincevincevince||4:27 pm Dec 22, 2005|
|Blocking domain specific links
||3||seasalt||11:33 pm Dec 21, 2005|
|Google appears to be ignoring robots file
No change after three months
|5||Lokutus||9:50 pm Dec 16, 2005|
|parameter page to be banned from crawling
|4||idolw||1:18 pm Dec 15, 2005|
|About the robots tag for Googlebot
Is it necessary?
|2||a1call||8:57 pm Dec 14, 2005|
|Remove part of your website
Should I use robots.txt or 301 redirects?
|1||twist||7:27 pm Dec 13, 2005|
|Robots.txt and the Google sandbox
Does de-indexing put one back into the sandbox?
|2||Rollo||7:05 pm Dec 13, 2005|
|Should I use the "?" or not.
||2||kgormat||3:52 am Dec 12, 2005|
|robots.txt and the law
do they intersect
|2||Powdork||10:19 am Dec 11, 2005|
|Newbie here require help
|2||sim20||11:36 pm Dec 2, 2005|
|disallow URL with parameters
/?id=asdf vs /
|5||Ratbert||11:18 am Dec 1, 2005|
|Ban Google from 1 directory
Ban Google from 1 directory
|2||dunhill||6:47 am Nov 30, 2005|
|Yahoo Ignoring robots.txt
Anyone know how to get Yahoo....
|2||marketingmagic||8:23 pm Nov 29, 2005|
How do I allow a page to be cached without right clicking allowed?
|3||NeedALife||7:44 pm Nov 29, 2005|
|Robots.txt not functioning!
Added the file months ago but results still coming up
|4||nwhorton||6:55 pm Nov 28, 2005|
|list of robots to disallow that actually obey robots.txt
||1||koen||3:32 pm Nov 28, 2005|
|does robots.txt really effect ranking?
||2||goneinthesun||4:44 pm Nov 26, 2005|
|Force spider to revisit sooner?
Robots instructions help.
|4||Fuzzi_Bear||2:32 pm Nov 26, 2005|
|Geting lots of hits on robots.txt but not much indexes in Yahoo/Google
||8||ball4121||4:47 am Nov 26, 2005|
|simple robots.txt question
just to make sure
|5||followgreg||4:24 am Nov 26, 2005|
|Robots.txt Vs META name="robots"
Which is better?
|4||Fuzzi_Bear||8:46 pm Nov 25, 2005|
|Quick robots.txt help
is this the right syntax?
|3||Fuzzi_Bear||2:17 pm Nov 25, 2005|
|ban all robots for a short period
||11||digicam||6:02 pm Nov 23, 2005|