Welcome to WebmasterWorld Guest from 54.145.208.64

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
parameter page to be banned from crawling
directory.asp?
4 idolw 1:18 pm Dec 15, 2005
About the robots tag for Googlebot
Is it necessary?
2 a1call 8:57 pm Dec 14, 2005
Remove part of your website
Should I use robots.txt or 301 redirects?
1 twist 7:27 pm Dec 13, 2005
Robots.txt and the Google sandbox
Does de-indexing put one back into the sandbox?
2 Rollo 7:05 pm Dec 13, 2005
Should I use the "?" or not.
2 kgormat 3:52 am Dec 12, 2005
robots.txt and the law
do they intersect
2 Powdork 10:19 am Dec 11, 2005
Newbie here require help
robots.txt
2 sim20 11:36 pm Dec 2, 2005
disallow URL with parameters
/?id=asdf vs /
5 Ratbert 11:18 am Dec 1, 2005
Ban Google from 1 directory
Ban Google from 1 directory
2 dunhill 6:47 am Nov 30, 2005
Yahoo Ignoring robots.txt
Anyone know how to get Yahoo....
2 marketingmagic 8:23 pm Nov 29, 2005
noimageindex?
How do I allow a page to be cached without right clicking allowed?
3 NeedALife 7:44 pm Nov 29, 2005
Robots.txt not functioning!
Added the file months ago but results still coming up
4 nwhorton 6:55 pm Nov 28, 2005
list of robots to disallow that actually obey robots.txt
1 koen 3:32 pm Nov 28, 2005
does robots.txt really effect ranking?
2 goneinthesun 4:44 pm Nov 26, 2005
Force spider to revisit sooner?
Robots instructions help.
4 Fuzzi_Bear 2:32 pm Nov 26, 2005
Geting lots of hits on robots.txt but not much indexes in Yahoo/Google
8 ball4121 4:47 am Nov 26, 2005
simple robots.txt question
just to make sure
5 followgreg 4:24 am Nov 26, 2005
Robots.txt Vs META name="robots"
Which is better?
4 Fuzzi_Bear 8:46 pm Nov 25, 2005
Quick robots.txt help
is this the right syntax?
3 Fuzzi_Bear 2:17 pm Nov 25, 2005
ban all robots for a short period
11 digicam 6:02 pm Nov 23, 2005
WebmasterWorld robots.txt
5 phpmaven 4:45 pm Nov 23, 2005
Mobile content
Mobile content might be seen as Duplicate.
1 designasp 12:41 am Nov 23, 2005
Gap - how many days
4 successlieswithme 9:39 am Nov 22, 2005
why has google not removed these url's
3 fzx5v0 3:48 am Nov 21, 2005
Capitalize robots.txt filename?
Robots.txt or robots.txt
3 4string 4:18 am Nov 17, 2005