Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Bot banning consultants and/or bad bot banning services Do they exist? Help for those of us who haven't a clue |
16 | Webwork | 3:00 am Sept 23, 2006 | ||||
| panscient data services block them NOW or pay later - 200 megs a day! |
7 | amznVibe | 5:11 am Sept 22, 2006 | ||||
| seo bot from AOL? who is this new bot showing up |
3 | bhartzer | 10:57 pm Sept 21, 2006 | ||||
| Bot or no Bot no user agent odd requests |
2 | Bewenched | 4:25 pm Sept 20, 2006 | ||||
| Google Mobile Indexing Agent? no request for robots.txt |
3 | keyplyr | 10:49 pm Sept 18, 2006 | ||||
| Adwords - adbot HELP - can't find it! |
2 | howiejs | 3:14 am Sept 15, 2006 | ||||
| IEAutoDiscovery Pre-fetcher, feed reader, e-mail harvester, or what? |
2 | GaryK | 8:03 pm Sept 13, 2006 | ||||
| Who is TridentSpider/3.1? spider from Slovenia? |
3 | Bewenched | 4:25 pm Sept 12, 2006 | ||||
| Why would ebay visit my site? |
7 | smokeybarnable | 6:28 am Sept 11, 2006 | ||||
| Microsoft with no UA Microsoft hitting site with no UA |
2 | ssgumby | 9:15 am Sept 7, 2006 | ||||
| ActiveTouristBot |
16 | Mokita | 12:08 am Sept 7, 2006 | ||||
| Fatlens |
3 | smokeybarnable | 5:37 pm Sept 6, 2006 | ||||
| MSN IP list |
3 | Brett_Tabke | 11:07 pm Sept 4, 2006 | ||||
| Googlebot visit pages that not exist googlebot is indexing pages that we dont have links |
3 | ivanff | 11:23 am Sept 2, 2006 | ||||
| What is Net::Trackback/1.01? -Good/Bad? |
5 | fusion5 | 1:57 am Sept 1, 2006 | ||||
| Filtering Internet noise: That is counting only human visitors I want to improve my own web logging tools |
10 | vite_rts | 12:58 pm Aug 28, 2006 | ||||
| Is Inktomisearch spider ADHD? Yahoo! Spider requests only 1 or 2 pages per visit. |
2 | geoffyp | 4:12 am Aug 28, 2006 | ||||
| problem with JSESSIONIDs and robot crawlers JSESSIONIDs combined with the use of Struts with Hibernate |
5 | KBee | 6:31 pm Aug 25, 2006 | ||||
| Getty Images spider? |
5 | Stu_Rogers | 7:02 am Aug 22, 2006 | ||||
| "libwww-perl/5.65" from an IBM address block scanning for blog feeds? |
2 | zCat | 9:42 am Aug 21, 2006 | ||||
| blogsearchbot-pumpkin No Robots.txt |
2 | Ocean10000 | 4:42 am Aug 21, 2006 | ||||
| New Russian Spider webalta.net |
9 | GaryK | 10:10 pm Aug 20, 2006 | ||||
| Found an IP range that I am banning Located it via Google Analytics |
10 | Mokita | 4:16 am Aug 19, 2006 | ||||
| china |
4 | topr8 | 3:24 pm Aug 18, 2006 | ||||
| Fast Meta Crawler |
4 | TheFlipguy | 4:02 am Aug 18, 2006 |
