homepage Welcome to WebmasterWorld Guest from 54.237.249.10
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Spider Spammer
Spammin' the UA
awoyo




msg:398265
 3:59 am on Jun 10, 2001 (gmt 0)

An (otherwise) unidentified bot came by and hit every page in the house, except of course for robots.txt, and hit them fast. The user agent was set to divulge the name of the product they sell, and the companies website URL. When I went to the site to check it out I found a site selling...stuff. It's the kind of stuff that I would be interested in since it's related to my site. In any other case I would give the UA here in the forum, but I don't want to give them any more play than they already got. It's amazing how creative people can be when it comes to internet marketing.

 

icehousedesigns




msg:398266
 12:38 pm on Jun 10, 2001 (gmt 0)

I would ban the whole domain.

Brett_Tabke




msg:398267
 12:47 pm on Jun 11, 2001 (gmt 0)

If you have good rankings, they were scrapping you for content most likely.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved