ZyBorg/1.0 Dead Link Checker

Constantly visits the site, why?

[spud01]

crawlers.looksmart.com (216.88.158.142)
13:52:41 on Tuesday, January 13, 2004.
Visitor used Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker ([e-mail]; [WISEnutbot.com).[...]

Should I be concearned receiving so many hits from this bot?

All pages have been checked and don't have any dead links.

[redzone]

Just Wisenut checking links on an index that rarely get's searched.....

[coyote]

That one has been poking around my site everyday for the past week.

I wonder why Looksmart is taking such an interest in checking and updating their links database as of late?

[spud01]

well its been hiting a site i maintaing for the past several months.

why does it take so long for the bot to get what it wants?

it there almost everyday.

[pendanticist]

I think before you all re-hash Zyborg's performance on the Internet yet another time, you might want to peruse this [google.com] string.

Well worth the read. :)

[spud01]

Thx

I came across this thread ==> [webmasterworld.com...]
msg #:2 seems to point out the following.

Judging from past updates I would say anywhere from 6 months to a year.

This is in response to how long it would take the crawl show up with Looksmart results.

Still don't know why the bot still revisits every day though, as the site doesn't have an infinite amount of pages to crawl and the revisits are to the same pages once a full crawl is complete. So...?

[WebJoe]

probably most recent thread about this: Looksmart/Zyborg & robots.txt [webmasterworld.com]