securing REST web services

[belfasttim]

Hi --

I built a bunch of PHP REST web services to provide data to some customers' websites, where they're basically syndicating some content.

It's not real high-security stuff (no credit cards etc.), but there are some confidentiality agreements in place that mean I must make a decent effort to protect the data being transmitted.

The service is just doing a SQL query in a PHP page and returning the data via HTTP.

I was thinking about a couple different ways to set up the security-- any input or suggestions most welcome.

1) Restrict access to the service by IP address (I know IP spoofing isn't that difficult, but like I said, it's not real high security stuff). This could be accomplished in PHP or Apache-- but I'm not sure if the IP address would be of the customer's server (that I know already) or someone browsing the customer's website. I think if the page calling the service was on the server, and not an actual page being rendered on the viewer's machine, it would work. (Please forgive me for not knowing how to explain this more clearly).

2) Make the user pass in a username/pass pair and do a lookup in the db before returning any data. This is easy enough to accomplish as well, but I'd rather not double the number of calls to the db if I don't have to. Also, I don't like the idea of hardcoding in the username/pass into the page that calls the service.

3) Using SSL to encrypt the transaction between the consumer and the service (I think this should be done regardless).

Like I said, I'm not too worried about someone saying they are a customer and consuming the "GET" services-- I'm more worried about someone using the "POST" services to change the data in the DB or break stuff.

I am leaning towards doing both-- restrict the services/ folder to a list of known customer IP addresses, and requiring username/pass for the services that add/update the database. That way the GETS (which will be far more frequently used) can stay slim and the POSTs can be more secure.

Any thoughts or tips?

Thanks

[coopster]

Definitely use SSL but also use a pre-shared key(s). If the pre-shared key(s) do not match, reject the request. Use a different pre-shared key per connection/client.

[httpwebwitch]

belfasttim, I'm struggling with the same thing right now. I have a RESTful API with access to sensitive (but not critical) data, and I want to limit access both horizontally and vertically, and I want it to be reasonably secure.

vertical access = which methods a user is allowed to access, what "kind" of data are they permitted to GET/POST/PUT/DELETE. Like, does the app (or the user) have admin privileges, write permissions, etc.

horizontal access = which datums are they permitted to interact with? Perhaps they can POST to their own data, GET from their friends, and if they try to see or molest anyone else's info they get the big 403 status slap.

The strategy I'm using is to issue a dev token and secret string, then ask for a hash of those along with each request. That covers the vertical access control, defining what functions the app can access. But then I want to know the context of the which user is using the app which accesses the API, so I also need a second identifier and hash for that.

There are two pieces of information that are secret, never sent in requests to the API. They are the developer's secret token, and the user's password. Both of these are salted up and hashed into an MD5

the ingredients:
1) a microtime(). This is merely an incrementing, usually-unique number
2) a dev token. This gives the requester permission to use the API
3) a hash, which is MD5($microtime . $devtoken . $secret)
4) a userid
5) a userhash, which is MD5($microtime . $userid . $userpassword)

So a complete request looks like (in GET)

GET
http://example.com/api/
?action=deleteuser
&param1=foo
&param2=bar
&microtime=8273984872
&devtoken=a1a1a1a1a1a1
&hash=a6df6d1g60d6fa5d4f5a432de8f7d
&userid=33
&userhash=bb89a6df6d1g60d6fa5d4f5a432d

When I receive a request to the API, the first thing I do is look up the $secret associated with that $devtoken. Then I check if MD5($microtime . $devtoken . $secret) == $hash. If it matches, then I know the developer token is valid and I can check permissions, do a little request throttling, vertical access control.

Then I check if MD5($microtime . $userid . $userpassword) == $userhash.

This whole thing is a little half-baked, and still in development.

The issues are:
1) I don't want to store user's passwords as plain text in my db, so I can't likely look them up to verify the $userhash
2) I'm not particularly keen on developers being able to write applications that collect login credentials. It's too phishy.
3) I'm currently not verifying that $microtime is unique, or incremental. Not sure that I want to, since it could spawn race conditions

So now I'm looking into something a little more complex, like setting up a session server, and issuing session tokens to the requesting app. Sessions that expire. Sessions you can only get if you come to http://example.com, fill out my form, then get redirected back to the app's landing URL with the session token attached.

I've been looking closely at how facebook does theirs. It is complicated.

Why is there no canonical according-to-hoyle step by step built-in solution for this? What is this, 2007? C'mon people, we need an open source API platform I can snap together like LEGO.

I'm interested in chatting up anyone who has built a solid XML/JSON hybrid API on LAMP, so I can avoid doing it wrong several times and wasting my precious life. Learning the hard way is for losers hahahaha.

[belfasttim]

Thanks for the input, httpwebwitch and coopster-- I did determine that it's trivial to restrict by IP address (since the way the request comes in means that it is indeed the server sending the request, not the browser), so that will be enabled immediately.

I like webwitch's idea of just hashing some pre-shared key info (as coopster was suggesting) and then using those to ensure that the request is coming from a trusted party.

webwitch, would you explain a little more about the microtime variable? How do both the consumer and the provider agree on this number, and where do they get it?

Thanks

[belfasttim]

after more thought and messing about, I think my half-baked but functional plan will be to hash together the user-id, their IP address, and a salt phrase, and give that to them as the api key. then on my end I hash the request's IP, the user-id (that they pass in as a parameter) and the top-secret salt phrase. This should give me pretty good control over access to the services.

I think by phrasing my queries right (eg. SELECT stuff, user-id WHERE user_active = 1) I can keep the inactive users from getting any results at all.

Thanks for the tips, and if you see anything really stupid please let me know. . .

[trillianjedi]

webwitch, would you explain a little more about the microtime variable? How do both the consumer and the provider agree on this number, and where do they get it?

Typically the time element is used to ensure that a request only lasts a certain amount of time. Otherwise, anyone that sees the HTTP request can just copy it and repeat it.

This is what Facebook do. I think essentially they extract the microtime variable and see if it's more than X seconds old. If it is, it gets rejected.

All servers relying on this need accurate clocks, so install NTPd if you want to use it.

Works quite well though.

[httpwebwitch]

Fbook goes even further: to use their API, you have to canonicalize the whole request along with its parameters, mix in a dev token, salt it with a secret, add microtime for noise, stir in a session and hash the whole shebang. It's really quite complex, all those hoops. Thankfully they offer a nice client package to do most of the work for you.

The part I dislike about my current strategy is sending a hash of the user's id + password. That means the requesting app is being trusted with the user's authentication creds - is that what I want? I think not.

Why am I daunted by the idea of issuing an expiring session token? Can I pull this off using PHP's built-in session_start() and $_SESSION, somehow?

[belfasttim]

Not sure if anyone is following this thread but I figured I'd post a followup question--

Does it make sense to initiate a session between the requesting server and the web service provider, since there will be multiple requests coming back and forth in any given time? It seems like it might simplify (or at least speed up) security checking, and it would also allow us to keep the database connection open between requests.

Any thoughts on this, pro or con?

[coopster]

Does it make sense to initiate a session

Only if your application calls for maintaining state.

it would also allow us to keep the database connection open between requests

HTTP is a stateless protocol so each request is made, processing occurs and a response is sent and the negotiation is then disconnected. There will be a new request to open a new database connection on every iteration.

[belfasttim]

thanks coopster-- you're right, I hadn't thought of that. thanks for the input.