Robino
msg:1525703 | 5:31 pm on Jan 2, 2004 (gmt 0) |
Looks like it.
|
ncw164x
msg:1525704 | 5:33 pm on Jan 2, 2004 (gmt 0) |
Yes you have stopped scooter from accessing your site A few examples of what you would put in your robots.txt file To exclude all robots from parts of the server
User-agent: *
Disallow: /cgi-bin/
Disallow: /misc/sitestats/ This indicates that nothing is disallowed and the spider can follow all links
User-agent: *
Disallow: To allow a single robot complete access and exclude all others
User-agent: Googlebot
Disallow:
User-agent: *
Disallow: / This would prevent your entire web site from being indexed
User-agent: *
Disallow: / hope this helps ncw164x
|
sem4u
msg:1525705 | 5:57 pm on Jan 2, 2004 (gmt 0) |
Yes it does. ncw164x has given you some good advice there about robots.txt
|
irock
msg:1525706 | 2:18 am on Jan 5, 2004 (gmt 0) |
Now that I have 'invited' Scooter. When will Altavista put me back on the index? Thanks!
|
ncw164x
msg:1525707 | 6:10 pm on Jan 9, 2004 (gmt 0) |
Hard to say, keep checking your site stats for any sign of scooter ncw164x
|
|
