XML Parsing Issue

While loop keeps going.....

[breham]

Hi, I've got a problem when parsing multiple xml feeds. The process should be: 1) get a list of feeds from db, 2) Parse the XML using simplexml, 3) put the results into another DB.

The code below works fine but when it's got to the last feed it starts again. There are about 400K rows of results before it starts again but I'm guess that's not too huge for mySQL to deal with.

Code:

include ("../functions/dbinfo.php");

$cntr=0;

//get the list of feeds
$sql2 = mysql_query("SELECT * FROM feeds");
$num_feeds = mysql_num_rows($sql2);


if ($cntr<$num_feeds) {
while ($row = mysql_fetch_array($sql2)) {

$feedName = $row['name'];
$feedNetwork = $row['network'];

$xml = simplexml_load_file ("../cache/".$feedName.".xml");
 
 //parse the feed based on feed format (parser also does insert)
 switch ($feedType) {

case "1":
include ("parsers/feed1.php");
break;

case "2":
include ("parsers/feed2.php");
break;
}

$cntr++;
}


}

Any suggestions what's wrong please?

[penders]

Where is $feedType being set?

Are there any other processes that insert rows into the 'feeds' table?

You should perhaps incorporate your $num_feeds check in your while() loop, since you are increasing the $cntr counter on every iteration of the loop, but not using it.

Replace this (removing your outer IF):

if ($cntr<$num_feeds) { 
  while ($row = mysql_fetch_array($sql2)) {

With this...

while (($cntr<$num_feeds) && ($row = mysql_fetch_array($sql2))) {

This might help, but I'm not sure why it would be required.

[rocknbil]

Try this too . . . and ask, why do you need a count of rows? I see this all the time - count the rows and do incrementing on a variable.

$num_feeds = mysql_num_rows($sql2);

If you do a while, it will only stay in that loop while it's stepping through the rows. No need to count, take out your incrementor and if and just do

include ($_SERVER['DOCUMENT_ROOT'] . "/functions/dbinfo.php"); 
//get the list of feeds 
$sql2 = mysql_query("SELECT * FROM feeds"); 
while ($row = mysql_fetch_array($sql2)) { 
 $feedName = $row['name']; 
 $feedNetwork = $row['network']; 
 $feedType=$row['feedType']; // See penders's Q 
 $xml = simplexml_load_file ($_SERVER['DOCUMENT_ROOT'] . " /cache/".$feedName.".xml"); 
 //parse the feed based on feed format (parser also does insert) 
 // If it's feed 1 and 2 and $feedType is 1 or 2, just use the variable. 
 // It will interpolate if double quoted. 
 include ("$_SERVER['DOCUMENT_ROOT'] . "parsers/feed$feedType.php"); 
} 

The only "if" I'd do is if $feedType is not 1 or 2, then I'd log the result.

include ($_SERVER['DOCUMENT_ROOT'] . "/functions/dbinfo.php"); 
//get the list of feeds 
$sql2 = mysql_query("SELECT * FROM feeds"); 
while ($row = mysql_fetch_array($sql2)) { 
 $id = $row['id']; 
 $feedName = $row['name']; 
 $feedNetwork = $row['network']; 
 $feedType=$row['feedType']; 
 if (($feedType==1) or ($feedType==2)) { 
 $xml = simplexml_load_file ($_SERVER['DOCUMENT_ROOT'] . "/cache/".$feedName.".xml"); 
 //parse the feed based on feed format (parser also does insert) 
 // If it's feed 1 and 2 and $feedType is 1 or 2, just use the variable. 
 // It will interpolate if double quoted. 
 include ($_SERVER['DOCUMENT_ROOT'] . "/parsers/feed$feedType.php"); 
 } 
 else { your_custom_logging_function("record ID $id has no feed type"); } 
} 

[breham]

Thanks both, the counter was just there out of frustration really, I knew it should work with just the while loop.

Anyway, I've taken out the if statement and implemented the " include ($_SERVER['DOCUMENT_ROOT'] . "/parsers/feed$feedType.php"); " statement instead of the switch and everythings working great now thanks.

Really appreciate your help, I spent too long trying to figure that out!

Brett

[penders]

Anyway, I've taken out the if statement and implemented the " include ($_SERVER['DOCUMENT_ROOT'] . "/parsers/feed$feedType.php"); " statement instead of the switch and everythings working great now thanks.

Glad you got it sorted. Just curious... presumably you set $feedType from your data as above? I'm still puzzled how these changes would prevent the loop from 'starting again'?