Welcome to WebmasterWorld Guest from 23.20.179.211

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

Trying to Open 712MB XML file

Please help me to open this

     
7:57 pm on Oct 30, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


Hello, thanks for all your support and help! This might sound stupid, but I am having trouble opening a 712MB XML file from Amazon.com. This is the smallest version that they offer and when I run my PHP script on a smaller file (25MB) it opens fine. By the way I'm using the SimpleXML function built into PHP 5. Yet when I try to open the larger file through the PHP script I get error messages after about 3-5 minutes about allocated memory and SimpleXML being unable to parse.

I just recently upgraded my computer's RAM to 1.25 GB but it doesn't seem to fix the problem (but did seem to help somewhat). I have no background programs running except the basics and Apache & MySQL too. Yet, it still won't open. I don't really have much money to upgrade my computer anymore at the moment. Is there anything that anyone could recommend to help? Thanks!

Keep coding!
Peace!

[edited by: CWebguy at 8:01 pm (utc) on Oct. 30, 2007]

8:50 pm on Oct 30, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


Check your php settings regarding max_execution_time [us3.php.net] and memory_limit [us3.php.net]. Both values would affect your ability to process a file that size.
10:02 pm on Oct 30, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


Yes, I put

set_time_limit(0);

in the script for the timeout factor. I'll see about the other one here in a minute.

10:24 pm on Oct 30, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


In the PHP.ini file the memory limit was set to 128MB, but that can't be right because whenever I run the script it uses the entire system's memory. Not only can I hear my computer chugging hard (kind of scary) but the memory availabe drops into the single digits :( . I tried upping the memory_limit to 700MB in the .ini file and restarted Apache but got same warning messages and results:

Warning: simplexml_load_file() [function.simplexml-load-file]: ....7770953: parser error : Memory allocation failed : xmlSAX2Characters ....on line 10

and a couple others. But like I said, it will do the smaller file which is in the same format so the script is not the problem. Is there anything else that can be done? Maybe a function in PHP or another setting that would work to open this large XML file and parse it?

Thanks!

10:34 pm on Oct 30, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:May 12, 2004
posts:533
votes: 0


things always get messy dealing with LARGE text files, have you tried splitting it and uploading it in chunks?
2:13 am on Oct 31, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


Thanks! Is there a way that you can recommend to split the XML file? (For instance using PHP or another method?)

Any help is greatly appreciated! Thanks!

4:18 am on Oct 31, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


By the way, the reason I ask is I'm having trouble opening in a text file (it just goes on and on loading). Maybe there is a way to split it without having to open and manipulate it. Thanks!
3:53 pm on Oct 31, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:May 12, 2004
posts:533
votes: 0


Any filesplitter will do the job, you've just got to be careful not to split it in the middle of a record.

I've got a good free one on my computer called "filesplitter" but for the life of me i can't find it on the web. but i'm sure there are plenty around.

4:43 pm on Nov 1, 2007 (gmt 0)

Preferred Member

5+ Year Member

joined:Aug 25, 2007
posts:531
votes: 0


Thanks! Just wondering, but why can't I split in the middle?

Thanks!

10:29 pm on Nov 1, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:May 12, 2004
posts:533
votes: 0


well if your record is

<name>fred</name>
<email>fred@fred.com</email>

and you've split the data then theres a possibility that

<email>fred@fred.com</email>

will become
<email>fred@

then in the next file
fred.com</email>

Which is fine if you're going to join the two back together again, but with a 700mb file you're far better to deal with each individual file and whack it in a database rather than rebuild the xml file and try and work with that.

11:17 pm on Nov 1, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 7, 2003
posts:1179
votes: 0


Try "Large Text File Viewer", it was created for viewing large 1GB text files
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members