homepage Welcome to WebmasterWorld Guest from 54.234.147.84
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Needle
Ouch, here is a thread for you.
blend27




msg:4566793
 12:36 pm on Apr 21, 2013 (gmt 0)

Tiny yet featureful HTTP client. With deflate & multipart POST support.

UA - Needle/0.4.8 (Node.js v0.8.22; darwin x64)

Headers:
Host: www.example.com
Accept: text/html,application/xhtml+xml,application/xml
Content-Length: 0
Connection: close

Single request from 108.180.197.18x.(TELUS CA). I am starting to see a few of these (JS Based)poking around .

 

lucy24




msg:4566866
 8:59 pm on Apr 21, 2013 (gmt 0)

Telus? Oh, ###. That's an utterly legitimate human IP.

Saw the thread title and though Woo Hoo, Pfui's back. Oh well.

dstiles




msg:4567123
 9:06 pm on Apr 22, 2013 (gmt 0)

I saw one a few days ago but was happy to see it blocked.

Ixquick returned only two real entries for the UA, both site logs (why do people allow that?!). Following an advert only returned a "this is popular, use it here" kind of page for cloud hosting.

I know telus is basically a humnan service but I have several bad groups and singles within their ranges.

lucy24




msg:4567138
 10:20 pm on Apr 22, 2013 (gmt 0)

:: idly wondering if a generic UA block in the form
[^.\d]0\.
or possibly
[/\s](v\.)?0\.
("version zero point anything")
would lock out anyone you didn't mean to lock out ::

keyplyr




msg:4567146
 11:16 pm on Apr 22, 2013 (gmt 0)



Not so bothered 'bout the needle, it's that impending haystack that's worrisome!

blend27




msg:4567166
 12:50 am on Apr 23, 2013 (gmt 0)

The point is that more and more AJAX based scarper scripts are out there being released into the wild. At this point HEADERS are very effective against that. Node.js is a serverside JS technology BTW.

keyplyr




msg:4567188
 4:51 am on Apr 23, 2013 (gmt 0)


Node.js is a serverside JS technology BTW.

Thanks for pointing that out. Sometimes I think all things js are browser-side.

lucy24




msg:4578126
 9:14 am on May 27, 2013 (gmt 0)

Ezra Pound.

2 down, 2 to go.

dstiles




msg:4578303
 6:54 pm on May 27, 2013 (gmt 0)

Have to say you are becoming obscure, Lucy. :)

lucy24




msg:4578328
 8:21 pm on May 27, 2013 (gmt 0)

See nearby post with similar timestamp :) This one was something about "generation of the thoroughly smug".

The snipped post gave a postcount of four, though I only ever saw two.

dstiles




msg:4578358
 9:19 pm on May 27, 2013 (gmt 0)

Fair enough. :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved