2 csv files comparison

csv files comparison

[fahad direct]

I am comparing 2 CSV files having his all new and old products inventory. How am i doing, i am reading his new csv file in data array by default php function and then comparing one by one each record with his old invenotry in his database and in case of 50,000+ records it is taking around 7 to 8 hours to be finished. Is there anything else i should do if i am doing in the wrong way?

[enigma1]

Use a unique identifier for each entry to speed up the search instead of comparing each column between the 2 files. Eg:

New CSV table
ID Model
5 ABC
7 DEF
3 GHI
etc

Old CSV table
ID Model
1 XYZ
2 JKL
3 GHI

Then you load and compare the csv ID columns only and you find the differences, store the different or same ids and then pull the records you need from the new CSV based on the ID column. I assume you only need the ones that don't exist in the old CSV. PHP has various functions for array manipulation (array_diff, array_diff_key etc).

The same you could do loading the files in a database and then use a single query to get the differences or identical records. You should be able to get the differences instantly.