Trying to Open 712MB XML file

Please help me to open this

[CWebguy]

Hello, thanks for all your support and help! This might sound stupid, but I am having trouble opening a 712MB XML file from Amazon.com. This is the smallest version that they offer and when I run my PHP script on a smaller file (25MB) it opens fine. By the way I'm using the SimpleXML function built into PHP 5. Yet when I try to open the larger file through the PHP script I get error messages after about 3-5 minutes about allocated memory and SimpleXML being unable to parse.

I just recently upgraded my computer's RAM to 1.25 GB but it doesn't seem to fix the problem (but did seem to help somewhat). I have no background programs running except the basics and Apache & MySQL too. Yet, it still won't open. I don't really have much money to upgrade my computer anymore at the moment. Is there anything that anyone could recommend to help? Thanks!

Keep coding!
Peace!

[edited by: CWebguy at 8:01 pm (utc) on Oct. 30, 2007]

[whoisgregg]

Check your php settings regarding max_execution_time [us3.php.net] and memory_limit [us3.php.net]. Both values would affect your ability to process a file that size.

[CWebguy]

Yes, I put

set_time_limit(0);

in the script for the timeout factor. I'll see about the other one here in a minute.

[CWebguy]

In the PHP.ini file the memory limit was set to 128MB, but that can't be right because whenever I run the script it uses the entire system's memory. Not only can I hear my computer chugging hard (kind of scary) but the memory availabe drops into the single digits :( . I tried upping the memory_limit to 700MB in the .ini file and restarted Apache but got same warning messages and results:

Warning: simplexml_load_file() [function.simplexml-load-file]: ....7770953: parser error : Memory allocation failed : xmlSAX2Characters ....on line 10

and a couple others. But like I said, it will do the smaller file which is in the same format so the script is not the problem. Is there anything else that can be done? Maybe a function in PHP or another setting that would work to open this large XML file and parse it?

Thanks!

[hughie]

things always get messy dealing with LARGE text files, have you tried splitting it and uploading it in chunks?

[CWebguy]

Thanks! Is there a way that you can recommend to split the XML file? (For instance using PHP or another method?)

Any help is greatly appreciated! Thanks!

[CWebguy]

By the way, the reason I ask is I'm having trouble opening in a text file (it just goes on and on loading). Maybe there is a way to split it without having to open and manipulate it. Thanks!

[hughie]

Any filesplitter will do the job, you've just got to be careful not to split it in the middle of a record.

I've got a good free one on my computer called "filesplitter" but for the life of me i can't find it on the web. but i'm sure there are plenty around.

[CWebguy]

Thanks! Just wondering, but why can't I split in the middle?

Thanks!

[hughie]

well if your record is

<name>fred</name>
<email>[e-mail]</email>

and you've split the data then theres a possibility that

<email>[e-mail]</email>

will become
<email>fred@

then in the next file
fred.com</email>

Which is fine if you're going to join the two back together again, but with a 700mb file you're far better to deal with each individual file and whack it in a database rather than rebuild the xml file and try and work with that.

[ncw164x]

Try "Large Text File Viewer", it was created for viewing large 1GB text files