is fsockopen faster than curl?

[httpwebwitch]

I need to send a GET request to a web service. Actually, I need to send lots of them - over 80 per second.

I've been using curl. but my benchmark testing shows that my app is too slow to handle the load.

tests on the web service independently show that it can handle hundreds of connections per second before it starts complaining. The bottleneck is in my app making the requests.

should I use fsockopen instead?
are they interchangable?

[enigma1]

You can do whatever you want with fsockopen and it will be a bit faster and a bit more complex for structuring requests.

But if you are saying is way slower, check if you using the CURLOPT_URL option for the same host to get multiple pages.

[httpwebwitch]

I switched from CURL to fsockopen. it is slightly faster, averaged out over dozens of tests, but the difference is negliglble. Less than 2%, and not a reliable 2% at that. (response time varies a lot depending on server load)

I'll stick with fsockopen because it works and it's slightly faster, but it wasn't the golden pill I was looking for

[trillianjedi]

Are you calling it by domain or IP address?

Zend may help speed things up if it's becoming CPU bound.

[andrewsmd]

Why do you need to send over 80 requests a second? Maybe there is another way to solve this problem.

[httpwebwitch]

@tj: calling by ip address, using an internal ip not a public one

@asmd: I need to send over 80 requests a second, because I am getting over 80 unique user interactions (peak) per second. It's not wasteful, it's just an extremely popular app. I want to optimize this server before I start scaling up with more hardware, since each clone will be just another copy of this one.

[Demaestro]

http... check out httplib

It is a pretty lightweight class. I haven't used it in PHP but I see there are several wrappers for it.

It is originally a python module, but I switched to it from pycurl, which is Python curl and I got a nice little performance boost. Hopefully that will be true in PHP as well.

[trillianjedi]

The bottleneck is in my app making the requests.

Have you managed to work out yet where it's bound? If it's CPU, I'd recommend compiling your PHP using Zend.

I can't see it being network bound - fsockopen in PHP I believe is a straight wrapper for the C libraries.

calling by ip address, using an internal ip not a public one

Wow. I'd expect you to be able to get a whole lot more than 80/second. What does the app do? Any external calls it makes itself after fetching the URL - eg hitting a database or writing output to disk?

tests on the web service independently show that it can handle hundreds of connections per second before it starts complaining

Do you mean socket connections or HTTP connections? At socket (TCP) layer, connections are extremely lightweight, but of course HTTP is quite a wordy protocol.

What socket options are you setting in your fsockopen call? Nagle algorithm in use or out?

If this is a dedicated box for this app, you might want to have a look at tuning the IP stack on the OS specifically for its purpose.

[httpwebwitch]

tj, you're awesome.

@Dmaestro, I'lll give httplib a shot and see if it outperforms fsockopen().

I did some more benchmark tests and found that most of the time is spent on a couple of database calls within the API which calls the web service, and loading & compiling the DAL. My DAL is a big class object that communicates with SQL via well named parameterized functions.

22% of the script execution time is spent defining my $DAL object, just so I can do something like $DAL->getuser($id);

I presume compiling the DAL with Zend would alleviate that. I've been doing PHP for many years, but I'm a Zend virgin. It may be time to pop that one.

MySQL Stored Procedures are another possibility. Using those, I wouldn't discard the DAL altogether, but its code would become dramatically smaller & simpler.

[trillianjedi]

Try switching the NAGLE algorithm off by setting the TCP_NODELAY socket option as part of your fssockopen routine. That will reduce a bit of latency, which won't be helping.

I presume compiling the DAL with Zend would alleviate that.

It might not alleviate it entirely, but it'll make a noticeable difference.

MySQL Stored Procedures are another possibility. Using those, I wouldn't discard the DAL altogether, but its code would become dramatically smaller & simpler.

My experience is stored proc's are also a bit quicker. No idea why and never found out how, but I have noticed....

[mercedes]

you may establish persistent connection to server, but i don't know => to do this/is this possible.

[jatar_k]

>> 22% of the script execution time is spent defining my $DAL object, just so I can do something like $DAL->getuser($id);

that smells like the insanity right there, maybe a new way to do that for this particular process.

>> httplib

my experience was that it was much slower than cURL

[Demaestro]

httplib

my experience was that it was much slower than cURL

Using Python I switched from curl (well really pycurl) to httplib for a SOAP exchange and I gained a performance boost.