homepage Welcome to WebmasterWorld Guest from 54.163.168.15
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Content, Writing and Copyright
Forum Library, Charter, Moderators: not2easy

Content, Writing and Copyright Forum

    
Website data extraction
Novus

5+ Year Member



 
Msg#: 4353417 posted 11:20 am on Aug 19, 2011 (gmt 0)

Has anyone had experience retriving data from a site using 3rd party software, i badly need an export of data from my site but do not have a facility to do so, im expecting my developers to charge but im wonding if there is some software that will crawl the site and pull off the info I (price, type, description etc) need for around 2000 products.

 

longen

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4353417 posted 12:23 pm on Aug 19, 2011 (gmt 0)

I've used the paid service from Mozenda but I assume you are looking for a free service.
If it is a SQL database could you not export the data yourself.

Novus

5+ Year Member



 
Msg#: 4353417 posted 1:19 pm on Aug 19, 2011 (gmt 0)

Hi I just looked a Mozenda, yes ist is an SQL database but its hosted with the developers which I have no access, My CMS is archaic so cant do it from there. Im expecting the devs to charge so i was just wondering if there is an alternative.

longen

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4353417 posted 2:08 pm on Aug 19, 2011 (gmt 0)

Compare the Developers fee with what Mechanical Turk would cost.

Novus

5+ Year Member



 
Msg#: 4353417 posted 8:41 am on Sep 21, 2011 (gmt 0)

Hi All

Just to re-touch on this subject my developers have increased there quote from 350 to 495 to 1125 for the task of creating a csv export facility from a CMS.

Due to the miss quoting they have offered to do the work for 750 now this still seems rather high considering exporting the data directly from the SQL server can be done relatively easly, The site is basic php, I would not thave thought it would be so expensive.

londrum

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4353417 posted 9:33 am on Sep 21, 2011 (gmt 0)

if all the info you need about each product is displayed using the same template, then it should be pretty easy to scrape the data straight from the page

if you're only talking about 2000 products then it wouldn't even take very long. you just need someone to write a simple little .php script for you

brotherhood of LAN

WebmasterWorld Administrator brotherhood_of_lan us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4353417 posted 9:35 am on Sep 21, 2011 (gmt 0)

A simple one line command with WGET can fetch all the pages, and as said, a simple script can extract the data from them.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Content, Writing and Copyright
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved