Welcome to WebmasterWorld Guest from 54.162.151.77

Forum Moderators: bakedjake

Message Too Old, No Replies

DuckDuckGo Raises $10 Million Amid Big Tech Privacy Fears

     
1:33 pm on Aug 30, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9911
votes: 972


DuckDuckGo, a pro-privacy search engine alternative, raised $10 million last week from Canada’s Omers’ venture capital fund amid growing privacy concerns from Google.

[breitbart.com...]
Part of the interest in DDG seems to be:
Douglas Schmidt, a research from Vanderbilt University, alleges to have discovered Google still has access to data of Google Chrome users browsing in private “Incognito” mode.


DuckDuckGo is beginning to gain traction. An alternative to g and b would be a fun thing to have.
2:03 pm on Aug 30, 2018 (gmt 0)

Senior Member from FR 

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Feb 15, 2004
posts:7139
votes: 412


An alternative to g and b would be a fun thing to have.

There is more than one alternative to g and b ..that does not track you already..
qwant
[lite.qwant.com...]
6:42 am on Sept 24, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


As soon as DuckDuckGo stops suplementing its SERP with Bing results, I'll start calling it a search engine.
7:18 am on Sept 24, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9911
votes: 972


Different strokes for DIFFERENT folks. :)
3:48 am on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


Today I saw the duck crawl approx 700 pages.Came from AWS using verified duck range.

It triggered one of my conditions and was blocked for requesting files too fast. It kept going getting 403s. Dumb duck.
4:28 am on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15704
votes: 811


Dumb duck.
Also watch out for its faviconbot--which should be encouraged because it makes your site more attractive in SERP--but which requires customized hole-poking due to a string of inexplicable behaviors.

I like DDG visitors. I don't see a lot of them, but they tend to stick around.
4:47 am on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


Yup, do that
10:31 am on Sept 26, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Sept 13, 2018
posts:355
votes: 71


For some reasons, i never see the DDG bot. I pay great attention not to block it, but there is no trace, while Google, Bing even Yahoo's slurp (!) are crawling tens of thousands of pages of my sites per day.

Same, for Qwant, I see, may be, one or two hits per month.
10:54 am on Sept 26, 2018 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3254
votes: 18


keyplr - what is the verified duckduck IP range, please?
11:01 am on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


Hi dstiles - you can either wait for me to get back to my desk late Thursday where I have my notes, or use the site search to find the couple of threads in the UA forum where we documented the 2 duck bots and their ranges.
3:39 pm on Sept 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2624
votes: 774


As soon as DuckDuckGo stops suplementing its SERP with Bing results, I'll start calling it a search engine.

I believe that DDG has stop using Bing results. But don't start calling it a search engine yet, now it uses Yandex results.

Full disclosure, I'm not sure if it has fully ended its relationship with Bing. ...checking...

I no longer find any reference to Bing or Yandex. At this point it is very unlcear.

But I did find this pro-tip from DDG (It is so good I felt it warranted its own thread) [webmasterworld.com...]
3:41 pm on Sept 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2624
votes: 774


Update. I went back and checked a link I had posted previously regarding the Yandex reference and the page that the link points to no longer has any mention of Yandex or Bing.
5:01 pm on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15704
votes: 811


what is the verified duckduck IP range
Have they got their own, now? I see the faviconbot from two specific AWS addresses, 54.208.102.abc and 107.21.1.abc (a single /32 each, in seemingly random alternation).

:: avidly awaiting the independent crawling bot, which I haven’t met yet ::
7:04 pm on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


The DDG crawl ranges I currently have as verified* is:

UA: Mozilla/5.0 (compatible; DuckDuckBot-Https/1.1; https://duckduckgo.com/duckduckbot)
Protocol: HTTP/1.1
Robots.txt: Yes
Host: AWS
50.16.0.0 - 50.19.255.255
50.16.0.0/14

However, AWS now being cloud distributed serving often will move instances to alternative nodes, so the IP range may vary within the AWS ranges in general.

Also: [webmasterworld.com...]

*verified when I emailed AWS and asked
7:11 pm on Sept 26, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


I'm seeing another strong crawl today: >2k files
10:24 am on Sept 28, 2018 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 14, 2008
posts:3254
votes: 18


Thanks, keyplr. I'll likely add duckduckbot to all amazon "whitelists" to be sure.
10:56 am on Sept 28, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


All AWS is best because it bounces around.