brotherhood_of_LAN - 10:38 pm on Oct 2, 2002 (gmt 0)

Just some smaller aspects that add up...

1. whitespace/spaces between html tags
2. length of urls...(thinking of those ones that span across the whole address bar)
3. robots.txt to keep some of the behaving spiders out of unwanted areas.
4. perhaps .htaccess/mod_rewrite (don't know much about these) to re-direct those bandwidth sucking bad spiders