I assume the "token" in this case just has to be either an encrypted string that the client can decrypt or some random string that gets stored somewhere (i.e. a database) that the client can then verify against, but I'm not really sure what the client is then supposed to do with the token or why you even need a token at all -- couldn't a simple user ID also suffice?
No - the token is a "ticket to ride." Just like a subway token. The client presents it to the gatekeeper when requesting service. In this case the provider does its own authentication, so the client presents the token back to the provider. In some cases the provider might delegate authentication - for example in the STS model, in which case the provider might hand the token off to a third party for authentication and even authorization.
From the service point of view, this token must:
- have a "shelf life". Otherwise, the token would be infinitely re-usable. So on the server-side, you can store the token in a session-based store, where you will get timeout for free. Or you can build a simple hashtable with expiration.
- be associated solely to the holder. In many cases the provider uses an approximation here and asserts that the token can be used only from the original requesting IP address.
So in step 3, the provider needs to check the username and password. If that validates, then create a token (hash), which refers to an entry in a Dictionary or Hashtable. The objects in the Hashtable are structures containing the username, the IP address, probably the original time-of-issuance, maybe the roles associated to the username, and whatever else you want to store. The service provider sends back this token - the hash, not the structure - to the client, typically as a Set-Cookie. When the client sends back the token (as a Cookie) on subsequent requests, the provider checks the dictionary to see if the token is available, has not timed out, matches the requesting IP address, is authorized for the requested resource, etc. If everything is ok, honor the request.
EDIT 2013 June
It's been several years now, and this answer is still getting votes. I'd suggest that people look into the OAuth 2.0 Framework, and Bearer tokens.
Basically they do just what is described here.
If you want a good example implementation, you can look at Apigee's Usergrid. It works this way:
User Authenticates
POST https://api.usergrid.com/token -d '{"username":"Joe","password":"Sec4et!","grant_type" : "password"}'
User receives an access_token in response
User makes subsequent calls with header Authorization: Bearer XXXXXXXXXX where XXXXX is replaced with the bearer token. This token has a Time-to-Live set by the usergrid server.
