Welcome to WebmasterWorld Guest from 54.144.124.152

Forum Moderated by: goodroi

Sitemaps, Meta Data, and robots.txt


Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.

 
Thread SubjectMessagesStarted byLast Message
Robot terms defined
The differences and similarities between nofollow, noindex and robots.txt
1 u_to_the_sixth 7:29 am Feb 26, 2005
Maximum size for robots.txt
Does any one know what it is?
6 blend27 9:40 pm Feb 25, 2005
robots.txt
robots.txt
4 indodpendo 8:59 pm Feb 25, 2005
How can I get my whole site out of search engines and Google cache?
2 mig1234 5:19 pm Feb 23, 2005
Banning Affiliate Links from being Indexed
1 bears5122 11:57 pm Feb 22, 2005
Advanced robots.txt question
About blocking subdomain on different server.
4 Buddha 12:57 am Feb 21, 2005
robots.txt wildcard matching
How wild is it?
8 kwasher 12:51 am Feb 21, 2005
Creating A Robots.Txt File
How Do I Create A Correct robots.txt File?
3 Jackal 4:27 pm Feb 14, 2005
ban all except index.htm
2 rebelo 3:08 pm Feb 12, 2005
Is it possible to block SSI for certain spiders?
Is it possible to block SSI for certain spiders?
1 mikeseo 3:19 pm Feb 10, 2005
Alexa (IA Archiver)
How do I excluse it
2 Rollo 3:21 am Feb 10, 2005
Disallow index.html file
adverse effects?
1 yowza 7:26 pm Feb 9, 2005
no robots.txt
6 textex 5:50 pm Feb 8, 2005
Please help me with robots.txt
8 karantiq 1:18 pm Feb 8, 2005
e-SocietyRobot
e-SocietyRobot new IP...
2 beauzero 5:47 pm Feb 4, 2005
Robot.txt
<meta name="Robots" content="index,follow">
4 Trojanfx 11:37 pm Feb 3, 2005
Can I Disallow Domain Hosts?
I want to disallow spiders from hosts, like foo.mydomain.com
1 SoCalSam 7:24 pm Jan 30, 2005
Robots.txt conundrum!
4 energylevel 6:36 pm Jan 30, 2005
Help I'm trying to get my site on search engines such as google
Help on how to get your site on search engines such as google
9 Mako3 8:32 am Jan 30, 2005
Correct way to exclude dynamic pages
question about differing methods of excluding dynamic pages
8 heuristick 6:50 pm Jan 28, 2005
are the robots.txt examples still valid from WebmasterWorld?
1 keywordguru 2:59 pm Jan 28, 2005
Suggestions On 'Low Value' Bots
Which bots provide little to no value to the web site.
2 netcent 6:24 pm Jan 24, 2005
Blocking a file in the main folder?
4 Ept103 3:33 am Jan 23, 2005
robot.txt needed to be crawled by google?
bot leaves after not finding one
8 thegreatpretender 4:25 pm Jan 22, 2005
Blocking sherlock
quick question to confirm format
11 Reno 2:03 am Jan 20, 2005

Featured Threads

Hot Threads This Week

Hot Threads This Month