Does anyone know the IP range of the info.com crawler? Seems to be hiding somewhere on AWS.
lucy24
9:52 pm on Jan 9, 2020 (gmt 0)
Cursory research indicates that info.com don't have a crawler of their own--I've never personally seen one--but instead aggregate data from other crawlers. Are you certain the one you've seen is legitimate?
:: pause for a moment of pity for the countries whose options include Qwant, whose robots I’ve had to block due to persistent misbehavior ::
ClosedForLunch
8:55 pm on Jan 10, 2020 (gmt 0)
I've only seen some referrers from info.com, but no identifiable crawlers.
