Forum Moderators: coopster

Message Too Old, No Replies

creating subfolders and skip on error feed extractor

hello, I bought feed extractor and currently it saves to ftp w/o subfolders

         

jz1977

4:50 pm on Oct 25, 2008 (gmt 0)

10+ Year Member



hello, I bought feed extractor and currently it saves to ftp without subfolders. This is a problem since it reaches the max file limit within one folder. I would like to control the saving function. would like some help.

this is the cron.php

#!/usr/bin/php -q
<?php
error_reporting(0);
set_time_limit(0);
include_once 'inc/config.php';

$ql = "SELECT * FROM feedext_feed WHERE active<>0 AND ftp_id<>'' AND DATE_ADD(lastrun,INTERVAL run_period HOUR)<NOW()";
$res = $db->Execute($ql) or die($db->ErrorMsg());
$f = 0;
$Repeat1__numRows = -1;

while (($Repeat1__numRows-- != 0) && (!$res->EOF)) {
$feed_url = $res->Fields("feed_url");
$amazon_id = $res->Fields("amazon_id");
$feed_title = $res->Fields("feed_title");
$feed_keyw = $res->Fields("feed_keyw");
$feed_descr = $res->Fields("feed_descr");
$feed_lng = $res->Fields("feed_lng");
$img_title = $res->Fields("img_title");
$img_url = $res->Fields("img_url");
$img_link = $res->Fields("img_link");
$img_w = $res->Fields("img_w");
$img_h = $res->Fields("img_h");
$tpl_num = $res->Fields("tpl_num");
$run_period = $res->Fields("run_period");
$subpath = '/httpdocs/archives/';
$pathto = '/var/www/vhosts/example.com/httpdocs/archives/';
$folder_name = date("Y/m/d/H")."/";
// $xml_fname = $res->Fields("id")."-".date("Ymd-His").".xml";
// $htm_fname = $res->Fields("id")."-".date("Ymd-His").".html";
$xml_fname = $res->Fields("id")."-".date("Y-m-d-H").".xml";
$htm_fname = $res->Fields("id")."-".date("Y-m-d-H").".html";
$xml_content = @file_get_contents($feed_url);
$xml_content = DoFeed((strpos($feed_url,'xml.amazon.com')?'amazon':''),$xml_content, $amazon_id, $feed_lng, $feed_title, $feed_keyw, $feed_descr, $img_title, $img_url, $img_link, $img_w, $img_h);
$xml_tmpfname = FLD_TMP."/".rndStr(32).".xml";
$handle = fopen(PATH_SITE.$xml_tmpfname, "w");
fwrite($handle, $xml_content);
fclose($handle);
$_REQUEST['XMLFILE'] = $xml_tmpfname;
$path = $pathto.$folder_name;
mkdir($path, 01777, true);
// echo $path;
// echo $htm_fname;
if($tpl_num){
$_REQUEST['TEMPLATE'] = $tpl_num.'.htm';
$html_content = GetHTMLPageContent();
$html_tmpfname = FLD_TMP."/".rndStr(32).".html";
$handle = fopen(PATH_SITE.$html_tmpfname, "w");
fwrite($handle, $html_content);
fclose($handle);
}
$ftp_id = $res->Fields("ftp_id");
$a_ftpids = explode(',',$ftp_id);
$ftpres = '';
foreach($a_ftpids as $ff){
$ftpres .= SendFtp($ff,$xml_fname,$xml_tmpfname,$htm_fname,$html_tmpfname)?'<span class=OK>OK</span><br />':'<span class=FAILED>FAILED</span><br />';
}

$db->Execute("UPDATE feedext_feed SET lastrun=NOW(),laststatus='$ftpres' WHERE id='".$res->Fields("id")."'");
$res->MoveNext();
@unlink(PATH_SITE.$xml_tmpfname);
@unlink(PATH_SITE.$html_tmpfname);
}
$res->Close();
?>

my problem is i can't figure out the right line of code to check folder if exists, and script ends on error or an xml error 9 invalid character.

in aplication.php the ftp function


function SendFtp($ftp_id,$xml_fname='',$xml_tmpfname='',$htm_fname='',$html_tmpfname='')
{
global $db;

if(($xml_fname && $xml_fname!='.xml') ¦¦ ($htm_fname && $htm_fname!='.html')) {
$ql = 'SELECT * FROM feedext_ftp WHERE id="'.$ftp_id.'"';
$res = $db->Execute($ql);
$a_ftp = $res->GetArray();
$a_ftp = $a_ftp[0];
if(!$a_ftp) return false;

$a_ftp['ftp_login'] = text_decrypt($a_ftp['ftp_login']);
$a_ftp['ftp_pwd'] = text_decrypt($a_ftp['ftp_pwd']);

$conn_id = ftp_connect($a_ftp['ftp_server']);
if ($conn_id==false) return false;

$login_result = ftp_login($conn_id, $a_ftp['ftp_login'], $a_ftp['ftp_pwd']);

if (!$conn_id ¦¦ !$login_result) return false;

ftp_pasv($conn_id, true);
if(!ftp_chdir($conn_id,$a_ftp['ftp_path'])) return false;
$upload1 = $upload2 = true;
if($xml_fname && $xml_fname!='.xml') {
$upload1 = ftp_put($conn_id, $xml_fname, PATH_SITE.$xml_tmpfname, FTP_ASCII );
@ftp_site($conn_id, "CHMOD 0666 {$a_ftp['ftp_path."/".date("Y/m/d/H")."/"']}{$xml_fname}");
}
if($htm_fname && $htm_fname!='.html') {
$upload2 = ftp_put($conn_id, $htm_fname, PATH_SITE.$html_tmpfname, FTP_ASCII );
@ftp_site($conn_id, "CHMOD 0666 {$a_ftp['ftp_path."/".date("Y/m/d/H")."/"']}{$htm_fname}");
}

if (!$upload1 ¦¦ !$upload2) return false;
ftp_close($conn_id);
}
return true;
}

using the date for subfolders seems like a resonable way to do this, each feed is updated every four hours and then made into a static file.

my question is, how can i make the cron complete each run even if there is an xml error in the feed. if error, skip feed, if folder exists, do not create folder.

any idea on how to make it work like this?

all files are ftp over.

warm regard,
-Jeff

[edited by: eelixduppy at 5:45 pm (utc) on Oct. 25, 2008]
[edit reason] removed URL [/edit]

Receptional Andy

5:52 pm on Oct 27, 2008 (gmt 0)



Hi Jeff, and welcome to WebmasterWorld :)

If this is a paid script, do you not get any kind of support from the author? You'll often get the best response there, or in the product's own support forum. It's quite tricky for those unfamiliar with a script to make any specific recommendations. In addition, if you modify a paid script, you may then be unable to keep it easily up to date with current versions.