Forum Moderators: open

Message Too Old, No Replies

Faster way for Data Load Infile, cvs

         

undream2

10:12 pm on Jun 15, 2009 (gmt 0)

10+ Year Member



Hi,

I developed a script to parse 16,000 records to be updated daily, from a MS spreadsheet - converted to CVS.. The script is basically done and works..

But, I ran into a new problem.. With the cvs file and database.

Some of the spreadsheet fields are blank, okay.. So, I couldn't upload the large files using big dump(unless I can configure the script for load data infile).. My only option to uploading the cvs with the blank fields.. Was using phpmyadmins load data infile..

Yes, all of this uploads just fine.. But, I can't upload all of the records at once.. And, I have to upload the file in sections.. --- which is very time consuming.. Especially, with MS spreadsheet..

And, I developed the script to be able to update the site daily, within minutes..

Is there a way that I don't have to upload the 16,000 records in sections, using data load infile.. Or, atleast a way to make the uploading in sections alot faster.. Anything, is alllot faster than scrolling down 16,000 records daily.. just to copy paste and upload the files...

thanks for any help.. and sorry for the long read.. I just really want this to work..

undream2

10:16 pm on Jun 15, 2009 (gmt 0)

10+ Year Member



also, I have even tried to select the 13 columns, as my sectioning upload method.. And, I can't get the individual field columns to get on the same record within the database.. When uploading it just creates more records, and rows.. And, doesn't overlap in the database..