Welcome to WebmasterWorld Guest from 22.214.171.124
Using a robots.txt is all part of being a good SEO. Be sure to check yours in the robots.txt validator that is available to subscribers.
|Thread Subject||Messages||Started by||Last Message|
Peerbot or I misunderstood the robot exclusion protocol
|3||WebJoe||9:11 pm Nov 5, 2004|
|Need help about File Robots.txt
how do I know spiders visit to my site or not?
|3||viva||10:15 am Nov 4, 2004|
|what happen with robot exclusion and iframes?
could it be a problem to exclude content that appears inside a home iframe?
|2||mgorricho||5:03 pm Nov 3, 2004|
|protocol extension resource control
proposed extension to robots.txt protocol allowing resource control
|1||copsewood||11:29 am Nov 2, 2004|
||2||papamaku||9:36 am Nov 2, 2004|
|robots and nav bars
excluding links in nav bars ...
|2||f00fighter||11:38 pm Oct 29, 2004|
|Need assistance with robot controls
need help with a robot .txt file
|2||borkboing||11:31 pm Oct 29, 2004|
|Robots.txt not working
bots not obeying robots.txt
|4||cyberjunkie||11:05 pm Oct 29, 2004|
|Use .htaccess to serve correct robots.txt?
Using htaccess and robots.txt to elimiate file duplication and 404 errors
|2||iceman||12:05 am Oct 26, 2004|
|Robots detection tool?
||2||obsos||1:17 am Oct 25, 2004|
|robots.txt vs index,follow
Who will win?
|3||brakkar||6:13 pm Oct 24, 2004|
|Removed 'deny all' robots.txt but no change
||2||kapow||11:03 pm Oct 23, 2004|
|What name does msn's spider use?
Lots of visits from yahoo and google....but not msn
|2||David_Bruning||7:05 am Oct 19, 2004|
|Robots.txt? (removing a specific subdirectory from google index...)
||3||hugo_guzman||8:06 pm Oct 18, 2004|
|More help in robots.txt
|2||sakuramy||1:25 pm Oct 18, 2004|
Error, Error, Error loads of them
|7||hatap||12:45 pm Oct 18, 2004|
|Allowing some bots, disallowing all others
||2||xbase234||5:16 pm Oct 16, 2004|
||5||Stra||4:41 pm Oct 16, 2004|
|Stop image bot from indexing my images
||2||ladyluck||1:17 pm Oct 14, 2004|
|Quick help with robots.txt please?
where to put the file?
|2||LadyTi||2:20 pm Oct 13, 2004|
|robots.txt to stop some domains
How to allow only one domain to be indexed for the same website.
|5||john_pinx||3:39 am Oct 12, 2004|
|duhhhhh, dumb blonde needing help with robot.txt
I don't have a clue - go easy please....
|4||ladyluck||6:37 am Oct 9, 2004|
|How to Disallow a Particular Page
||5||jsnively||9:25 pm Oct 6, 2004|
|hacking robots.txt for RSS features?
||1||phildog||3:53 pm Oct 4, 2004|
|Is robots.txt always needed?
|10||benwalsh||12:38 pm Oct 4, 2004|