Forum Moderators: phranque

Message Too Old, No Replies

What IP range does Duck Duck Go search bots use?

Might need to white-list them in my IP blocking list

         

SumGuy

1:03 pm on Dec 12, 2019 (gmt 0)

5+ Year Member Top Contributors Of The Month



Does anyone know what IP range is used by Duck Duck Go's search bots? (I assume they operate search bots). I never see anything that looks like their search-bot user-agent in my web logs so I might be blocking them. Actually, same goes for yahoo search bots also. I haven't seen them in a long time. I hope they don't use AWS. I block a lot of AWS IP range.

brotherhood of LAN

1:31 pm on Dec 12, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



[help.duckduckgo.com...]

They use AWS.

I'd doubt you'd get de-indexed by blocking them anyways, as it's Bing results (and Yandex in some countries). Their bot grabs favicons and home pages to find those favicons.

phranque

4:02 pm on Dec 12, 2019 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



this thread from our Search Engine Spider and User Agent Identification forum might be helpful:
DuckDuckBot [webmasterworld.com]

lucy24

6:03 pm on Dec 12, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Their bot grabs favicons and home pages to find those favicons.
Caution: Unlike some faviconbots one could name, ddg’s will only get the favicon if its preceding request for the front page was not blocked. (My current RewriteRules have two separate exemptions to deal with this issue, because sending autoreferers for the front page is such a common malign-robot tactic.)