cURL form post with receipt verification and timeout repost

php cURL form post url mysql response verification timeout repost

[razn8]

I need help in PHP/MySQL to retrieve a response from a cURL url data post.

I'm inserting form data to a local mysql table, then post the form data to a URL for insertion into the client's a mssql server. In the event of a time out, I need to retrieve a response from the remote server to verify the post has been received. If a response is received, the record in my local mysql table is flagged as delivered, If no response is received, the record data is reposted to the url until a response is received.

I have successfully tested the url post using cURL. What I need help with is retrieving the response, and flagging the record to archive or repost based upon whether a response is received.

Here is my code so far:

<?php
// Connects to your Database
mysql_connect("localhost", "username", "password") or die(mysql_error());
// select database.
mysql_select_db("dbname") or die(mysql_error());

// Get values from form
$VendorID = 'username';
$Type= 'new_record';
$firstName = $_POST['firstName'];
$lastName= $_POST['lastName'];

// Insert data into mysql
mysql_query("INSERT INTO names VALUES ( '$VendorID', '$Type', '$firstName', '$lastName' )");

// send to url

//extract data from the post
extract($_POST);

//set POST variables
$url = 'http://255.255.255.0/WebTransfer/';
$fields = array(
'VendorID'=>urlencode($VendorID),
'Type'=>urlencode($Type),
'firstName'=>urlencode($firstName),
'lastName'=>urlencode($lastName)
);

//url-ify the data for the POST
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);

//execute post
$result = curl_exec($ch);

//close connection
curl_close($ch);

?>

[PHP_Chimp]

I am going to be particularly annoying and not actually answer your post. My excuse is a lack of time...
However

$firstName = $_POST['firstName'];
$lastName= $_POST['lastName'];

I just need to say NO! and NEVER!. Even if this is just code to test, people who are less experienced will look at that and think that is the way to do it.
You never accept user supplied data without cleaning it. At least pass it through mysql_real_escape_string [uk2.php.net].

[razn8]

I may not have clearly asked this question. Let me restate the question:

I'm inserting form data to my local mysql table, then I post the form data via cURL to a client's remote url.

I need help with recording a response into my local MySQL table that the client's remote server has received the cURL form post. If a response is not received from the client's remote server, I need to resend the form data post every 10 minutes until a response is recieved. Once a response is received I need to record it in my local MySQL table entry.

So I think I need to figure out the following:

How do I capture and insert the response receipt into the corresponding MySQL table entry?
How do I check for the response record every ten minutes and repost the form data if no response is recieved?

[razn8]

I will try to clarify my question again.

I am guessing that, along with my form mysql data fields, I need to create a Received field. If a remote response from the url form post is received, the Received field is updated to Yes, and the entry is complete. If there is no response from the remote server, the Received field is updated to No. If the Received field value is No, the data for that entry is reposted to the remote url. If a response from the repost is received, the Received field is updated to Yes, and the entry is complete. If the repost does not generate a response from the remote server, the Received field is not updated and this process will repeat in ten minutes, or whatever time delay I specify.

Any recommendations?

[coopster]

First, heed the warning in regards to user-supplied data. You need to validate and properly format it before you use it or store it. This is paramount.

Next, to answer the questions ...

How do I capture and insert the response receipt into the corresponding MySQL table entry?
How do I check for the response record every ten minutes and repost the form data if no response is recieved?

You can set curl to return the response headers and then you can use curl functions to read them. The response headers will tell you whether or not you were successful. If your response comes back unsuccessful you handle accordingly, mark the status in your sql table record or whatever you are doing to handle responses. In order to trigger a repost I suppose you could create a crontab that reads this file for "unsuccessful" status and try to post again. You may want to store a "number of attempts" column too and then maybe email yourself or something to let you know that you are not having any success after "x" number of attempts.

[razn8]

coopster,
Thanks! That is very helpful.

Now all I need to do is figure out how to set curl to return the response headers, use curl functions to read them and mark the status in my mysql table record as Successful or Unsuccessful.

Then I need to figure out how to create a crontab that reads status and posts the form data to the url if it is marked as Unsuccessful.

Storing a "number of attempts" column and sending an email if the number of unsuccessful attempts exceeds a specified amount is something else I will want to figure out. Thanks for that idea.

I used curl_getinfo() function and curl_errno() function to print the following:

Array ( [url] => http://www.example.com/ [content_type] => text/html [http_code] => 200 [header_size] => 159 [request_size] => 264 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 0 [total_time] => 0.057687 [namelookup_time] => 0.016364 [connect_time] => 0.016512 [pretransfer_time] => 0.016519 [size_upload] => 63 [size_download] => 1246 [speed_download] => 21599 [speed_upload] => 1092 [download_content_length] => 1246 [upload_content_length] => 0 [starttransfer_time] => 0.057619 [redirect_time] => 0 ) 0

Can I use this to read the response? How would I translate it and insert the Successful record into the mysql table? Would an if else function be used to mark it either Successful or Unsuccessful?

[edited by: coopster at 6:31 pm (utc) on Feb. 16, 2009]
[edit reason] please use example.com, thanks! [/edit]

[coopster]

You are welcome, and welcome to WebmasterWorld, by the way.

The Successful response code of 200 is an OK response. RFC2616 [ietf.org] has all the details of HTTP and although somewhat difficult to read/understand the first time I would encourage any developer to have it handy for reference.

You check the value of the response and handle accordingly in your application logic.

[razn8]

coopster, thanks again.

So if I receive a code of 408 (request timeout), then it would be unsuccessful. If I receive a code of 200, it is successful. That sounds like a good place to start? Then figure in the other codes in some way.

Any chance you can point me towards a source that will help me produce the necessary application logic that I've described:

1.Marking the mysql table as Successful or Unsuccessful based upon the response headers.
2.Creating a crontab to repost the data if the response is Unsuccessufl

By the way, I'm formatting the user-supplied data like:

$firstName = mysql_real_escape_string(stripslashes($_POST['firstName']));

[coopster]

Marking the mysql table? I assume you have a column defined in the table which you will maintain to display either success or no success based on the curl response. You just insert your row with the current status for that column and then update it when necessary.

You can

man crontab

from your server command line or if you do a quick search for crontab you'll find lots of instruction on how to use the command.

[razn8]

I tried this after the curl_exec to grab the http code and insert it into the mysql table, but it didn't work. Am I close?

//execute post
$result = curl_exec($ch);

// Get values from response header
$httpCode = curl_getinfo($ch, curlINFO_http_CODE);

//show information regarding the request
print_r(curl_getinfo($ch));
echo curl_errno($ch) . ‘-’ . curl_error($ch);

//close connection
curl_close($ch);

// Insert response data into mysql
mysql_query("INSERT INTO names (`response`) VALUES ('$httpCode')");

[coopster]

Looks like you are getting closer. The CONSTANT is case-sensitive:

$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

[razn8]

I got the http code to insert, but my form insert values changed to null. I want to insert the form data before I send the curl post, then insert the http response code. Do I need to set an ID in the first insert then specify that ID for the second insert? Or is there a simpler method? Heres my code:

<?php
// Connects to your Database
mysql_connect("localhost", "username", "password") or die(mysql_error());
// select database.
mysql_select_db("dbname") or die(mysql_error());

// Get values from form
$VendorID = 'username';
$Type= 'new_record';
$firstName = mysql_real_escape_string(stripslashes($_POST['firstName']));
$lastName= mysql_real_escape_string(stripslashes($_POST['lastName']));

// Insert data into mysql
mysql_query("INSERT INTO names VALUES ( '$VendorID', '$Type', '$firstName', '$lastName' )");

// Select data from mysql
$data = mysql_query("SELECT * FROM names")
or die(mysql_error());

// print table on page to view data for testing
Print "<table border cellpadding=3>";
while($info = mysql_fetch_array( $data ))
{
Print "<tr>";
Print "<th>VendorID:</th> <td>".$info['VendorID'] . "</td> ";
Print "<th>Type:</th> <td>".$info['Type'] . "</td> ";
Print "<th>First Name:</th> <td>".$info['firstName'] . "</td> ";
Print "<th>Last Name:</th> <td>".$info['lastName'] . " </td></tr>";
}
Print "</table>";

// send to url

//extract data from the post
extract($_POST);

//set POST variables
$url = 'http://www.domain.com/';
$fields = array(
'VendorID'=>urlencode($VendorID),
'Type'=>urlencode($Type),
'firstName'=>urlencode($firstName),
'lastName'=>urlencode($lastName)
);

//format the data for the url POST
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);

//execute post
$result = curl_exec($ch);

// Get values from response header
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

//show information regarding the request
print_r(curl_getinfo($ch));
echo curl_errno($ch) . ‘-’ . curl_error($ch);

//close connection
curl_close($ch);

// Insert response data into mysql
mysql_query("INSERT INTO names (`response`) VALUES ('$httpCode')");

?>

[coopster]

I would run the curl first, receive the response, then insert the new row including the response status into the database table. By the way, no need to select the data from the database again when you already have all the values you just entered in -- just display them again to the user if that is your intention.

[razn8]

I'm printing the data to test that it is inserted into the table. I will not be showing this to the user.

My local insert is a backup in the event that the remote post times out. I will try what you suggested

Thanks!

[coopster]

You can monitor the remote post and upon failure handle accordingly.

[razn8]

Do you recommend a better way to monitor the remote post than what I am currently attempting. I ran the curl first then inserted to my local table and it worked. I received responses 200 for the good test and 403 for the failed test where I removed the receiving page. How can I simulate a time out and receive a 408?

[razn8]

coopster,
I hope I'm not annoying you, but I really appreciate your advice. Before I setup a crontab, I'm making the script to check for the httpCode and resend the curl post. I'm taking baby steps, checking the response entry for a 200 value, else I resend the curl post. Now I need to figure out how to get the values from the 'else' rows for the curl post. What would be the best way to do that?
Heres my code:

// Connects to your Database
mysql_connect("localhost", "username", "password") or die(mysql_error());
// select database.
mysql_select_db("dbname") or die(mysql_error());

// Select data from mysql
$qry = mysql_query("SELECT httpCode FROM tablename")
or die(mysql_error());

while($data = mysql_fetch_array( $qry ))
{

{
if( $data['httpCode'] == 200)
{
echo "200<br />";
}
else
{

// send to url

//extract data from the post
extract($_POST);

//set POST variables
$url = 'http://domain.com/';
$fields = array(
'VendorID'=>urlencode($VendorID),
'Type'=>urlencode($Type),
'FirstName'=>urlencode($FirstName),
'LastName'=>urlencode($LastName)
);

//format the data for the url POST
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);

//execute post
$result = curl_exec($ch);

// Get values from response header
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

//close connection
curl_close($ch);

}
}
}

[coopster]

No, you're not annoying me. If I don't respond for any given period of time it's because I'm working away. I drop in and contribute here in the forums when I am able.

OK, I guess what we are looking at now is after the initial curl post and you are writing some code that ultimately will end up in a crontab. So, you loop through the rows in the database ... where ultimately you will likely run a query with a WHERE clause something along the lines of:

$qry = mysql_query("SELECT <column options list here> FROM tablename WHERE httpCode <> 200")

I think that is what you are showing anyway. Well, what you need to do is pull the POST data from the table rows where you first stored it upon the initial attempt to post the form data. It failed. So, you must keep it somewhere, right? In your table. Grab that post form data back out of the table and try again. That is what you do in the "else" part here.

[razn8]

Thanks. What I'm doing is querying the field httpCode, which stores the response from the curl post. If the field value is 200, it prints "200 and a line break", just for my testing confirmation, else it runs the curl post again, which enters blank values into the mysql because I have figured out how to pull them out yet.

So far I'm thinking that mysql_fetch_assoc might do it? Here's where I left off before dinner:

// Select data from mysql
$qry = mysql_query("SELECT httpCode FROM tablename")
or die(mysql_error());

while($row = mysql_fetch_array( $qry ))
{

{
if( $row['httpCode'] == 200)
{
echo "200<br />";
}
else
{

while ($row = mysql_fetch_assoc($qry)) {
echo $row["VendorID"];
echo $row["Type"];
echo $row["FirstName"];
echo $row["LastName"];
}

This may not be the right way, but if it is, I need some help figuring out how to set these values in the curl post fields array.

[razn8]

If I'm going to need to pull all the row data from each failed row, repost it, then update the httpCode if the repost receives a 200, do I need to query unique ID fields?

[coopster]

In order to attempt a POST once again later on you are going to need to have access to the data that is required to make the post. If you are doing so in the very same script in the very same instance that the first post was attempted, great -- you still have the original $_POST variables available in memory. If not, you need to set up a statement that reads the data from the table (WHERE original response code is not 200 OK) and attempt to post the data again.

[razn8]

Thanks, my code 2 posts preceding does exactly that. If the code is 200, it prints 200, else it runs the curl again. What I need to figure out is, when it is not 200, what is the statement to get the field data for that row and send it to the curl post variables?

[coopster]

Yes, I understand. And you are correct, and that's what I'm trying to explain. My thoughts are along these lines ...

All posts have been attempted. It is two days later. We decide to run a script that looks for unsuccessful post attempts and repost. Call our script, which executes the following sql query statement based on the fields you are showing in your table ...

$sql =  
 "SELECT 
    VendorID, 
    Type, 
    FirstName, 
    LastName 
 FROM tablename 
 WHERE 
    httpCode <> 200" 
; 
$rows = mysql_query($sql); 
while($row = mysql_fetch_array($rows)) { 
  // Fill your CURL POST fields with this information, 
  // just like you did on the first post attempt 
  $fields = array( 
    'VendorID' => urlencode($row['VendorID']), 
    'Type' => urlencode($row['Type']), 
    'firstName' => urlencode($row['FirstName']), 
    'lastName' => urlencode($row['LastName']) 
  ); 
  // Now use curl to attempt the POST again 
} // And now we return to our loop to attempt a POST on the next one ... 

[razn8]

Thanks a ton! That worked great! Now when I repost the data, I need to retrieve the new response and update that in the mysql row, so if repost sends back a 200, it won't send the data again the next time the script runs.

Is it something like this:

//execute post
$result = curl_exec($ch);

// Get values from response header
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

//close connection
curl_close($ch);

// Update httpCode in mysql
mysql_query("UPDATE tablename ( `httpCode`) VALUES ( '$httpCode' )");

Do need to get this into the Update somehow?: while($row = mysql_fetch_array($rows))

------------------------

Here is my code that worked for the repost:

// Connects to your Database
mysql_connect("localhost", "user", "password") or die(mysql_error());
// select database.
mysql_select_db("dbname") or die(mysql_error());

// Select data from mysql
$rows = mysql_query(
"SELECT
VendorID,
Type,
FirstName,
LastName
FROM tablename
WHERE
httpCode <> 200"
);
while($row = mysql_fetch_array($rows)) {

// Fill CURL POST fields with this information,


//set POST variables
$url = 'http://domain.com/';
$fields = array(
'VendorID' => urlencode($row['VendorID']),
'Type' => urlencode($row['Type']),
'FirstName' => urlencode($row['FirstName']),
'LastName' => urlencode($row['LastName'])

);

//format the data for the url POST
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
rtrim($fields_string,'&');

//open connection
$ch = curl_init();

//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch,CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch,CURLOPT_SSL_VERIFYPEER, false);

//execute post
$result = curl_exec($ch);

// Get values from response header
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);

//close connection
curl_close($ch);

} // return to loop to attempt a POST on the next unsuccessful row ...

[razn8]

coopster, I figured out the update. Thanks so much for the help. By showing me the first half, I was able to figure out the second half.

I just added this update statement after closing the curl:

// Update httpCode in mysql
$row = mysql_query("UPDATE tablename SET httpCode='$httpCode'" )
or die(mysql_error());

Now all I need to do is figure a crontab to run the repost automatically and I'm done.

Thanks again!