Uploading 12MB csv file to MySQL taking 25 minute

30,000 records, surely this is not ideal

[centime]

Hi All

I am updating an applications product data base

Shared linux hosting

0.5M upstream broadband connection

I believe these are hobby level facilities, but I cannot claim to be a pro till i can truely make a living at this.

Should i be able to do this faster, and would it be worthwhile getting improved hosting, or is this a tech set up isssue

Thanks

[lorax]

There may be a limit on the upload by the Host or interface. PHPMyAdmin has some limit (2.XMB I think). Try doing a smaller amount of data. If more than one table, do one of the smaller ones. Keep the file sizes to no more than 1MB and I suspect you'll do fine.

[centime]

I read a thread where chaps where talking about uploading 700,000 plus records to MySQL in about 2 minutes

This is why i am a bit perturbed, plus its a pain, multiplied

[lorax]

Understood. It could just be a limitation of the host. Have you contacted their tech support to ask?

[stuartc1]

Could be a few things.... if you are loading the data using a third party tool (phpmyadmin, sql front end tool) then it can take much longer. Try doing it directly using the >mysql -uuser -p -d database < file.sql type command, this usually runs much much faster. If this is the method you are using, then you may need to ask the host to increase some allocated resources to your account.

Actually.. I noticed you mentioning you broadband speed.... this suggests you have the data stored on your local machine, and loading the data remotely to mysql. This is a bad idea.... first FTP the file to the server then run it. That way you are not restricted to your connection speed.

[centime]

Hi Stuart, uploading to the server an then importing to MySQL would be great.

But Phpmyadmin only browses my local pc to find files to import

I am not yet up to speed with LAMP systems , so how would I get my csv into mysql from an ftp folder in my hosting account ?

I know php can be confiured to run sql commands from a text file, but would rather not have to convert a massive csv to test sql commands

Cheers

[phranque]

your problem is using the http protocol to upload large amounts of data.
it is not an efficient protocol for the task.
in a recent test i made under ideal conditions i was able to upload at about twice the speed you see, so your results are typical.
with ftp/sftp you might upload that file in 5 or 10 minutes since they are "file transfer protocols".

[centime]

Thats correct, but once I upload the csv file, how do i get it into MySQL ?

The hosting company only provides phpmyadmin

Can phpmyadmin be persuaded to import a file from a folder inside the hosts server, that is my webspace on the hosting server