homepage Welcome to WebmasterWorld Guest from 107.21.187.131
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
sv-fw.looksmart.com
addvisors




msg:397113
 5:31 pm on Oct 31, 2003 (gmt 0)

What is this.sv-fw.looksmart.com

I appeared today, but does looksmart use a spider?

Best regards

 

addvisors




msg:397114
 6:53 pm on Oct 31, 2003 (gmt 0)

Btw. Hello everybody, this is my first post in here.

Look forward to spending more time in here.

Kim

pendanticist




msg:397115
 7:23 pm on Oct 31, 2003 (gmt 0)

Welcome to WebmasterWorld addvisors. :)

Actually, Looksmart has two of them:

(http://www.WISEnutbot.com is a search engine.)

Their newest Dead Link Checker.

216.88.158.142 - - [29/Oct/2003:15:58:55 -0800] "GET /robots.txt HTTP/1.1" 200 1524 "-" "Mozilla/4.0 compatible ZyBorg/1.0 Dead Link Checker Beta-d01 (wn.zyborg@looksmart.net; http*//www.WISEnutbot.com)"

and

64.241.242.18 - - [17/Oct/2003:23:20:59 -0700] "GET /robots.txt HTTP/1.1" 200 1524 "-" "Mozilla/4.0 compatible ZyBorg/1.0 (wn.zyborg@looksmart.net; http*//www.WISEnutbot.com)"

Sorry you had to bump your own 'first post' :(

Normally, answers are lots faster in coming.

Pendanticist.

addvisors




msg:397116
 7:32 pm on Oct 31, 2003 (gmt 0)

Thanks a lot for the answer,

Does that mean that the wisenutbot will eventually crawl my site when it has visited my index.htm now?

Best regards,

Kim

pendanticist




msg:397117
 7:54 pm on Oct 31, 2003 (gmt 0)

It has been my experience that once a (legitimate) bot visits my robots.txt file (even if only one visit has been made as you mentioned), eventually that bot will return and begin the indexing process.

You might find this thread [webmasterworld.com] interesting reading as it has links to specific posts where various bots (good/bad and evil) are pretty much all mentioned. The most current is at the bottom of that page.

Pendanticist.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved