Forum Moderators: open
They do NOT take requests or submissions and it is a very detail oriented search engine.
Too bad I didn't find out the same until I denied Jack.
He hasn't returned since.
I have the URL somewhere.
If I find it I'll add it back later
Some times we can be too over bearing :-(
Most of the "minor" bots I see only take robots.txt and those pages that are listed in the ODP. There's a steadily growing number of link checkers out there, which isn't necessarily a bad thing.
Hey bird,
link checkers can be a useful tool when WE apply that use to our own websites.
Somebody else using a link checker to travel my site is IMO not desireable.
If only a link or two that is accepatable to me.
When I see a link checker at the beginning's of traveling my entire sites that offers a different consideration.
IMO extesnive link checking on my sites by another is no diifferent than the many research and other bots which have no clear definition on "intent of use."
...then that's not what I call a link checker. The link checkers I was talking about try to make sure that their links really exist and serve a page. For that purpose they have to fetch those of your pages they link to.
Spiders that crawl full sites are a completely different story. I'm not saying that those don't exist, but at least in my stats they are a minority (by total numbers, not necessarily by numbers of accesses).
