Welcome to WebmasterWorld Guest from 22.214.171.124
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
|duhhhhh, dumb blonde needing help with robot.txt
I don't have a clue - go easy please....
|4||ladyluck||6:37 am Oct 9, 2004|
|How to Disallow a Particular Page
||5||jsnively||9:25 pm Oct 6, 2004|
|hacking robots.txt for RSS features?
||1||phildog||3:53 pm Oct 4, 2004|
|Is robots.txt always needed?
|10||benwalsh||12:38 pm Oct 4, 2004|
||4||rhodopsin||11:40 am Oct 3, 2004|
|Insert disallow.php to robots
is it a good idea?
|3||sirkei||5:52 am Oct 3, 2004|
|Can i add robot txt to a free hosting site? keywords
||2||alze||5:14 am Oct 3, 2004|
|Does robots.txt prevent all access?
...on page as well as directory.
|2||yump||5:02 am Oct 3, 2004|
Common? How to protect?
|4||adfree||11:05 am Sep 30, 2004|
|newbie to robots.txt, want to exclude /images/
excluding some image folders but not others
|2||franco81||3:29 am Sep 30, 2004|
|Newbie - Why wouldn't I want to allow certain robots?
If I don't have anything on the site I want to keep private
|4||AmyG||4:03 pm Sep 29, 2004|
|What if your domains robots.txt
Was pointed to your domain?
|5||Daily_Sparring||3:22 am Sep 27, 2004|
|Where to put robots in website?
I has doubt where to put robots.
|3||viva||10:00 am Sep 26, 2004|
|robots.txt and .htaccess
||1||HuhuFruFru||12:02 am Sep 26, 2004|
|Excluding Sub Folders
When you exclude a folder, are folders inside that excluded also?
|3||damlag||8:39 pm Sep 25, 2004|
|Can someone help me write a robots.txt?
||5||budbiss||11:50 pm Sep 19, 2004|
How to avoid spidering of anything with this prefix
|2||nuevojefe||10:26 pm Sep 19, 2004|
|5||wolfline||5:10 am Sep 17, 2004|
robot.txt working against me?
|5||seofan||9:36 pm Sep 8, 2004|
|who can give me a website log file sample to see?
website log file
|3||Tellist||5:19 pm Sep 4, 2004|
|Google CHTML Proxy disobeying robots.txt
|5||yosmc||12:16 pm Aug 31, 2004|
|Google tells me to change my robots.txt
Should I follow the advice from Google adSense?
|2||Bonusbana||2:44 pm Aug 27, 2004|
|Master List of Malicious Robots
Is there an active list of the bots we should block?
|5||Eric_Lander||4:09 pm Aug 25, 2004|
|robots.txt security problems?
||2||helohelo||3:44 pm Aug 25, 2004|
|301 Redirect domain?
How to handle robots.txt?
|5||Mr_Roberto||1:13 am Aug 24, 2004|