Welcome to WebmasterWorld Guest from 54.242.105.196

Forum Moderators: open

How do I get search engine results programatically?

   
8:28 pm on Nov 23, 2012 (gmt 0)



How do I get search engine results programatically? Some search engines use AJAX to display content and so the URL does not change. This makes it difficult to get result content from a search engine programatically.

Most search engines do not use AJAX. But two popular ones that I am interested in is YANDEX and DUCKDUCKGO. Both of these search engines use AJAX to advance beyond the first page of results. How do I find a way to programatically access these pages?
2:52 am on Nov 24, 2012 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



welcome to WebmasterWorld, complete!

you might want to familiarize yourself with the terms of service for the search engines in which you are interested.
they typically frown on scraping search results using "automated methods".

some search engines provide an API which may be sufficient for your requirements:
DuckDuckGo API - [duckduckgo.com...]
Yandex.XML - [api.yandex.com...]
 

Featured Threads

Hot Threads This Week

Hot Threads This Month