Welcome to WebmasterWorld Guest from 54.197.116.116

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

Trying to Open 712MB XML file

Please help me to open this

   
7:57 pm on Oct 30, 2007 (gmt 0)

5+ Year Member



Hello, thanks for all your support and help! This might sound stupid, but I am having trouble opening a 712MB XML file from Amazon.com. This is the smallest version that they offer and when I run my PHP script on a smaller file (25MB) it opens fine. By the way I'm using the SimpleXML function built into PHP 5. Yet when I try to open the larger file through the PHP script I get error messages after about 3-5 minutes about allocated memory and SimpleXML being unable to parse.

I just recently upgraded my computer's RAM to 1.25 GB but it doesn't seem to fix the problem (but did seem to help somewhat). I have no background programs running except the basics and Apache & MySQL too. Yet, it still won't open. I don't really have much money to upgrade my computer anymore at the moment. Is there anything that anyone could recommend to help? Thanks!

Keep coding!
Peace!

[edited by: CWebguy at 8:01 pm (utc) on Oct. 30, 2007]

8:50 pm on Oct 30, 2007 (gmt 0)

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Check your php settings regarding max_execution_time [us3.php.net] and memory_limit [us3.php.net]. Both values would affect your ability to process a file that size.
10:02 pm on Oct 30, 2007 (gmt 0)

5+ Year Member



Yes, I put

set_time_limit(0);

in the script for the timeout factor. I'll see about the other one here in a minute.

10:24 pm on Oct 30, 2007 (gmt 0)

5+ Year Member



In the PHP.ini file the memory limit was set to 128MB, but that can't be right because whenever I run the script it uses the entire system's memory. Not only can I hear my computer chugging hard (kind of scary) but the memory availabe drops into the single digits :( . I tried upping the memory_limit to 700MB in the .ini file and restarted Apache but got same warning messages and results:

Warning: simplexml_load_file() [function.simplexml-load-file]: ....7770953: parser error : Memory allocation failed : xmlSAX2Characters ....on line 10

and a couple others. But like I said, it will do the smaller file which is in the same format so the script is not the problem. Is there anything else that can be done? Maybe a function in PHP or another setting that would work to open this large XML file and parse it?

Thanks!

10:34 pm on Oct 30, 2007 (gmt 0)

10+ Year Member



things always get messy dealing with LARGE text files, have you tried splitting it and uploading it in chunks?
2:13 am on Oct 31, 2007 (gmt 0)

5+ Year Member



Thanks! Is there a way that you can recommend to split the XML file? (For instance using PHP or another method?)

Any help is greatly appreciated! Thanks!

4:18 am on Oct 31, 2007 (gmt 0)

5+ Year Member



By the way, the reason I ask is I'm having trouble opening in a text file (it just goes on and on loading). Maybe there is a way to split it without having to open and manipulate it. Thanks!
3:53 pm on Oct 31, 2007 (gmt 0)

10+ Year Member



Any filesplitter will do the job, you've just got to be careful not to split it in the middle of a record.

I've got a good free one on my computer called "filesplitter" but for the life of me i can't find it on the web. but i'm sure there are plenty around.

4:43 pm on Nov 1, 2007 (gmt 0)

5+ Year Member



Thanks! Just wondering, but why can't I split in the middle?

Thanks!

10:29 pm on Nov 1, 2007 (gmt 0)

10+ Year Member



well if your record is

<name>fred</name>
<email>fred@fred.com</email>

and you've split the data then theres a possibility that

<email>fred@fred.com</email>

will become
<email>fred@

then in the next file
fred.com</email>

Which is fine if you're going to join the two back together again, but with a 700mb file you're far better to deal with each individual file and whack it in a database rather than rebuild the xml file and try and work with that.

11:17 pm on Nov 1, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Try "Large Text File Viewer", it was created for viewing large 1GB text files