Welcome to WebmasterWorld Guest from 184.108.40.206
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|robots & blog
||2||zuses||2:54 pm Sep 22, 2006|
|Keeping robots out of cgi bin
What's the best format?
|10||Reno||6:00 pm Sep 21, 2006|
|What user agent does Overture use?
when they ping your page for quality
|2||Bewenched||8:35 pm Sep 20, 2006|
||2||Northstar||7:50 pm Sep 20, 2006|
|Need Help With Robots.txt Question
||4||BlueLeaf||4:23 am Sep 19, 2006|
|Contrary advice over robots.txt validity
Who is right Google or the other?
|3||netchicken1||1:05 pm Sep 18, 2006|
|Disallow page disallows variables, too?
example.php and example.php?id=10
|3||4string||8:50 pm Sep 17, 2006|
||3||Boogalou||4:07 am Sep 17, 2006|
|Disallowing access to feed URLs
Should I disallow access to feed and trackback URLs
|1||kektex||4:45 am Sep 16, 2006|
|Exclude all robots on alias domains?
Good idea or not?
|1||jtara||5:52 pm Sep 15, 2006|
|Strange way of writing robots.txt
Has anybody use this?
|2||Cyclob||3:04 pm Sep 15, 2006|
Error issued by Sitemaps
|5||apprentice||7:43 pm Sep 9, 2006|
|IS there any tool to find out the particular page got spidered?
|3||geobals||8:50 pm Sep 7, 2006|
|Remove www. before sitename
||4||tigersoft||5:14 pm Sep 7, 2006|
|I think google can't access my pages.
||6||rj87uk||10:16 am Sep 4, 2006|
|Ban IP range
||3||hostlead||9:18 pm Sep 2, 2006|
|Signature Spammer Bot?
A pest on forums.
|1||helleborine||1:43 pm Aug 29, 2006|
||1||dave1236||1:51 pm Aug 28, 2006|
|Possible with robots.txt?
Possible with robots.txt?
|2||Aforum||9:22 am Aug 28, 2006|
|Do i need robot.txt on my site
robot.txt-is it needed & how to add it
|5||Amitbud||1:46 pm Aug 25, 2006|
|Prevent spider from following external links
to keep Amazon items out of search engine database
|2||Reno||7:11 am Aug 25, 2006|
|Robots txt to remove pages
Does google still look at urls blocked
|3||cheaperholidays||7:56 am Aug 24, 2006|
|Where do I find the Robot Validation Tool?
robot validator validation robots
|7||myebux||9:45 pm Aug 23, 2006|
|Need advice on my robots.txt file
new to robots.txt, can someone check this file
|4||jwa55121||5:07 pm Aug 22, 2006|
|Are there any tricky uses of robot.txt file?
robot.xt file to be indexed by SEs?
|4||yuvika||8:47 am Aug 22, 2006|