Forum Moderators: open

Message Too Old, No Replies

Kroger.com 'webcrawlers'

         

Pfui

1:21 am on Feb 19, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



webcrawler01.kroger.com
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; MS-RTC LM 8)

robots.txt? NO

Requested a wrongly titled page => 404. Never seen again. Not 100% sure it was a bot per se; possibly an outbound server for employees? (Search results show numerous, non-spam posts by multiple posters via the same Host.) If the latter, an unfortunate sub-domain naming scheme.

FWIW:

webcrawler01.kroger.com = 158.48.6.nn
webcrawler02.kroger.com = 158.48.6.nn
webcrawler03.kroger.com = 158.48.133.nn
webcrawler04.kroger.com = 158.48.133.nn

[edited by: incrediBILL at 1:53 am (utc) on Feb 19, 2010]
[edit reason] Obscured IPs [/edit]

System

5:00 am on Feb 23, 2010 (gmt 0)

redhat



The following 2 messages were cut out to new thread by incredibill. New thread at: search_engine_spiders/4085421.htm [webmasterworld.com]
12:43 am on Feb 23, 2010 (PST -8)

frontpage

9:46 pm on Mar 29, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My IDS just banned this thing (unknown if bot or human) automatically due to the high number of connections.

webcrawler04.kroger.com using these two user agents within seconds of each other.

"Mozilla/4.0 (compatible;)"


"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MS-RTC LM 8)"