homepage Welcome to WebmasterWorld Guest from 50.17.7.84
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Faxobot/1.0
coming soon per home page
Busynut




msg:399794
 1:31 am on Sep 14, 2004 (gmt 0)

Hi all,

Noticed a new visitor at my site today - the website seems to be modelled after google - but none of the links work (as of earlier this evening).

Referer: [faxo.com...] Agent: Faxobot/1.0

 

fiestagirl




msg:399795
 4:00 pm on Sep 14, 2004 (gmt 0)

Yes, seen them coming around.

No info on the site.
No link to info in the UA.
No requests for robots.txt.
Ip's resolve to the isp only - no info there.

Not off to an auspicious start.

idiotgirl




msg:399796
 4:38 am on Sep 16, 2004 (gmt 0)

Hit me today, too. Seems to go straight for the goods without asking for robots.txt. Rude, or just anxious?

Web site looks like another Google knock-off.

bull




msg:399797
 4:09 pm on Sep 28, 2004 (gmt 0)

It was here today and requested robots.txt after sucking "/".
Banned it,too.

MirageOne




msg:399798
 8:28 pm on Oct 13, 2004 (gmt 0)

Hi All,

I ,too, noticed this site today. I set up a bot trap on my site specifically to capture and ban robots that do not follow the robots.txt standard.

The trap sent the following string via e-mail:
A bad robot hit (snipped URL) 2004-10-13 (Wed) 06:54:51 address is 69.152.89.194, agent is Faxobot/1.0

Now if you visit the above mentioned folder you will be banned automatically by my site so please don't click on the link above. ;)

Here's the article that I used to set up the trap, well worth the read for any Web Admin. Examples scripts include a PHP method and a .htaccess method.

[kloth.net ]

It will automate the process of banning bad bots from your site.

John

[edited by: volatilegx at 2:00 pm (utc) on Oct. 14, 2004]
[edit reason] removed URL [/edit]

Lord Majestic




msg:399799
 10:38 am on Oct 14, 2004 (gmt 0)

It was here today and requested robots.txt after sucking "/".

Was the time difference between two requests very small (1 second or so), or not?

pendanticist




msg:399800
 6:54 am on Nov 25, 2004 (gmt 0)

Maybe I can answer that since this new scurge just tripped my trap, ran thru 44 files before switching IP Numbers and beginning the crawl again rather successfully. Such to say, it did NOT trip the trap the second time around! All without requesting robots.txt.

The IP Number switch took exactly two seconds.

The banned crawl in which it was being fed 403s, ran between two and eight seconds per file.

After the IP Number switch ( From 24.107.33.4 to 69.155.184.142 ), it ran at roughly the same speed, only this time it requested the same file ( over and over again ) as many as six times over a span of one minute.

All in all, I'd say a ballpark average would be one file ever 2-3 seconds.

ignatz




msg:399801
 1:42 am on Dec 18, 2004 (gmt 0)

I'm seeing Faxobot, any new info on banning? ip ranges, etc?

Lamb




msg:399802
 6:12 pm on Dec 21, 2004 (gmt 0)

I just got Faxobot, it seems like a person:
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:57:34 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:57:39 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:57:45 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:57:50 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:57:56 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:58:05 AM
adsl-69-155-4-253.dsl.stlsmo.swbell.net (Faxobot/1.0) Requested ... On 12/13/04 At 9:58:11 AM

And the list goes on. They were all different pages though.

pendanticist




msg:399803
 6:50 am on Dec 22, 2004 (gmt 0)

69.155.30.206 - - [21/Dec/2004:14:28:07 -0800] "HEAD /Blahblah.html HTTP/1.0" 403 0 "http://MyRootURL.com/" "Faxobot/1.0"
69.155
.4.253 - - [21/Dec/2004:14:28:14 -0800] "HEAD /Blahblah.html HTTP/1.0" 200 0 "http://MyRootURL.com/" "Faxobot/1.0"

Tripped the trap with the first IP Number. Once it got tired of being fed 403s, it switched IP Numbers ( again ) continuing on without incident.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved