Welcome to WebmasterWorld Guest from 54.234.153.186

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

Import CSV zipped file date to mysql

     

pemba76

4:00 pm on Feb 11, 2010 (gmt 0)

5+ Year Member



How do i import a CSV file that is zipped.
CSV file = xyz.txt
Zipped file = xyz_zip.zip

I have worked on the following code to get the whole xyt.txt date into a variable ($content).

<?php
$zip = zip_open("xyz_zip.zip");

if ($zip)
{
while ($zip_entry = zip_read($zip))
{
if (zip_entry_open($zip, $zip_entry))
{
$contents = zip_entry_read($zip_entry,zip_entry_filesize($zip_entry));
echo "$contents<br />";
zip_entry_close($zip_entry);
}
}
zip_close($zip);
}
?>

The problem is that fopen() need a CSV file but i have is a variable with the content of the CSV file.

How do i import $content data into mysql database ?

Please help.

jatar_k

4:34 pm on Feb 11, 2010 (gmt 0)

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I don't understand why you need to use fopen if you already have the contents in a var

why not just insert the content of that var into the db then?

pemba76

5:01 pm on Feb 11, 2010 (gmt 0)

5+ Year Member



I am quite new to php.
How do i insert the content of a variable into the db.
The contents is records like below :
Name|Address|Age
aaa|bbb|22
ddd|yyy|45
eee|fff|56
I have to insert Name, Address, Age into the db from the var.

jatar_k

7:19 pm on Feb 11, 2010 (gmt 0)

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member



ah I get it now

it looks like there is a new line, you could split it on the newline character, which would return an array

you could then loop through that array and split each row on the | and then insert those pieces into the db

pemba76

4:23 pm on Feb 12, 2010 (gmt 0)

5+ Year Member



Got it to work finally.
Now i have another problem. The zip files that i have to import is large (something is GB's). This is causing the memory to get exhausted.
Is there a way of splitting these zip files in PHP ?

jatar_k

6:06 pm on Feb 13, 2010 (gmt 0)

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member



just don't read the whole thing into memory, unzip it first or read it line by line

pemba76

2:34 pm on Feb 14, 2010 (gmt 0)

5+ Year Member



do u mean to say unzip the file xyz_zip.zip and read the content of xyz.txt and create a new file xyz_copy.txt and the read from xyz_copy.txt line by line.
i am not sure but is there a way in php to unzip the file xyz_zip.zip and save the xyz.txt as a seperate file or do i have to write the data to a new file(xyz_copy.txt ) and save it ?
please note that the zipped version of the file itself is in GBs and this is causing memory to get exhausted.

jatar_k

2:09 pm on Feb 15, 2010 (gmt 0)

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member



trouble is, if your files are too big for this particular function, then they're too big

maybe using a shell script would work to unzip and chop it up

though if it gets unzipped then you can work through it with php, though you may have to extend max_execution_time

massive files are always a problem, it may take more than one language/script to deal with it
 

Featured Threads

Hot Threads This Week

Hot Threads This Month