Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

How do I get search engine results programatically?

8:28 pm on Nov 23, 2012 (gmt 0)

New User

5+ Year Member

joined:Feb 14, 2011
votes: 0

How do I get search engine results programatically? Some search engines use AJAX to display content and so the URL does not change. This makes it difficult to get result content from a search engine programatically.

Most search engines do not use AJAX. But two popular ones that I am interested in is YANDEX and DUCKDUCKGO. Both of these search engines use AJAX to advance beyond the first page of results. How do I find a way to programatically access these pages?
2:52 am on Nov 24, 2012 (gmt 0)


WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
votes: 195

welcome to WebmasterWorld, complete!

you might want to familiarize yourself with the terms of service for the search engines in which you are interested.
they typically frown on scraping search results using "automated methods".

some search engines provide an API which may be sufficient for your requirements:
DuckDuckGo API - [duckduckgo.com...]
Yandex.XML - [api.yandex.com...]