Welcome to WebmasterWorld Guest from 126.96.36.199
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|Exclude all robots on alias domains?
Good idea or not?
|1||jtara||5:52 pm Sep 15, 2006|
|Strange way of writing robots.txt
Has anybody use this?
|2||Cyclob||3:04 pm Sep 15, 2006|
Error issued by Sitemaps
|5||apprentice||7:43 pm Sep 9, 2006|
|IS there any tool to find out the particular page got spidered?
|3||geobals||8:50 pm Sep 7, 2006|
|Remove www. before sitename
||4||tigersoft||5:14 pm Sep 7, 2006|
|I think google can't access my pages.
||6||rj87uk||10:16 am Sep 4, 2006|
|Ban IP range
||3||hostlead||9:18 pm Sep 2, 2006|
|Signature Spammer Bot?
A pest on forums.
|1||helleborine||1:43 pm Aug 29, 2006|
||1||dave1236||1:51 pm Aug 28, 2006|
|Possible with robots.txt?
Possible with robots.txt?
|2||Aforum||9:22 am Aug 28, 2006|
|Do i need robot.txt on my site
robot.txt-is it needed & how to add it
|5||Amitbud||1:46 pm Aug 25, 2006|
|Prevent spider from following external links
to keep Amazon items out of search engine database
|2||Reno||7:11 am Aug 25, 2006|
|Robots txt to remove pages
Does google still look at urls blocked
|3||cheaperholidays||7:56 am Aug 24, 2006|
|Where do I find the Robot Validation Tool?
robot validator validation robots
|7||myebux||9:45 pm Aug 23, 2006|
|Need advice on my robots.txt file
new to robots.txt, can someone check this file
|4||jwa55121||5:07 pm Aug 22, 2006|
|Are there any tricky uses of robot.txt file?
robot.xt file to be indexed by SEs?
|4||yuvika||8:47 am Aug 22, 2006|
|New spider that is actually good.
found a "good" new spider
|1||jcrimi||12:10 am Aug 22, 2006|
problem disallowing pages in home directory
|2||surfpro203||2:23 am Aug 16, 2006|
|Flash web site - need for robots.txt?
I have a web site in Flash - is it necessary to have a robots.txt file?
|2||Cratima||1:05 pm Aug 15, 2006|
|Https? Where'd that come from?
Site dropped like a rock
|2||Dragonlady26||1:56 am Aug 12, 2006|
|Please double check my robots.txt
||1||lee_sufc||9:30 am Aug 7, 2006|
Is the robots.txt file absolutely necessary
|5||camchoice||6:57 pm Aug 6, 2006|
|'robot.txt' file to be indexed by SEs?!
are there any tricky uses of robot.txt file?
|2||yuvika||6:56 pm Aug 6, 2006|
|Disallow index.html in Robots.txt
Is it safe to disallow an index.html file of a website
|3||pixeladdict||8:48 pm Aug 4, 2006|
|disallow every 'filename' except.
disallow every 'filename' except...
|2||decbrad||4:12 am Aug 4, 2006|