PHP/MSSQL Massive data insert

What would be the best approach?

[milocold]

Hi fellas,

I'm trying to find the most efficient method to use in order to insert 5 to 10 thousand records. Basically, I have a script that creates pins; the user just selects the number of pins to create.

I have to believe that something better than this exists:

#open DB

#while loop to perform a crazy amount of mssql inserts

#close DB

Anybody have any ideas or creative solutions? =)

Currently, I'm doing research to find out if I can dynamically create an excel file (containing all the pins) and push that data into the mssql DB via php. I don't think it's possible though...=(

Thank you in advance!

M.Cold

[Steerpike]

That sounds, ummm, wildly inefficient. Can I ask exactly what you're trying to do? and what your database/tables are trying to store? Maybe there's a better way of organising the tables so that you don't need to do so many inserts.

[milocold]

I do agree that the method I posted sucks, but it's the only thing I've found in relation to performing bulk inserts from dynamically generated data.

I don't think that I have access on the DB server to create stored procedures, but if I do that will be the route I'll be taking. Even though, at the end of the day, the same while-loop methodology will be applied...I think. =/

Oh, and the data being stored (in a single table) are the combination of consecutive numbers, random numbers, and constants.

Thanks,

M. Cold

[penders]

I've been away from the DB scene for a while, but we used to be able to do bulk inserts from a text file in Sybase SQL. Essentially one INSERT statement that read the contents of a file (CSV, TAB seperated etc.) Without the overhead of multiple INSERTS and continuous index building.

A bit of a google... does your version of MSSQL support this sort of thing? (Is there a 'bcp' - bulk copy program?)

BULK INSERT tblMyTable FROM 'c:\mydata.txt' WITH (FIELDTERMINATOR = ',')

May be the database forum [webmasterworld.com] can offer more help...?

[milocold]

Thank you Penders!

I think your right on the money. I was thinking DTS package or bulk insert (from excel to db), but a CSV would be much better.

*hits himself on the head*

Plus, I'm not even sure you can work with DTS packages from PHP. I know you can using ASP/.Net platform. At any rate, thank you again. I'm gonna go perform the test now! =)

M. Cold