Welcome to WebmasterWorld Guest from 220.127.116.11
I am trying to write a crawler which will enable me to scrape dynamic webpage.As I am a PHP developer , I am trying to see if it can be done with PHP
Can anybody give me some pointers to proceed?Also I would like to know if there is any other technology/technique with which it can be done.
Thanks in advance.
Thanks for your reply.
Is there any way to deal with AJAX content ? for example on page load some JS function is called and that function fetches data using AJAX and adds it to innerHTML of a DIV.
Google can't make that guarantee for all sites on the web.
EDIT: You should also look into SSJS [en.wikipedia.org]; I'm not sure whether it could be hooked up to PHP.
EDIT2: You don't happen to run your site on a PC do you? Darn. [addons.mozilla.org]
EDIT3: Eureka! I was searching "Gecko" but needed "spidermonkey." And it looks like Pooh Bear might be involved too: [devzone.zend.com...]
[edited by: Jesdisciple at 6:29 pm (utc) on June 19, 2009]