Shibboleth and SAML

Shibboleth is an open source web single sign-on product which is very popular with universities around the world. It is primarily an implementation of the Security Assertion Markup Language (SAML), but supports other identity protocols as well. SAML v2 defines a specific type of identifier that may be used to identify a user within a transaction known as a Persistent Identifier. The name itself does not convey it’s full meaning, but it is defined as “a persistent opaque identifier for a principal that is specific to an identity provider and a service provider or affiliation of service providers.” This is SAML’s directed identity.

Shibboleth’s implementation of persistent identifiers has matured a bit over the years, but the main algorithm remains the same. At it’s simplest, the identifier is simply a hash of three things: an identifier for the user, an identifier for the identity provider, and an identifier for the relying party. Using md5 as our hashing algorithm, and using a few made up values, this would look like:

md5( "jsmith" + "http://idp.example.com/" + "http://sp.example.com/" ) = 
    2f6bc52c7527747eff263f4183c7f402

When a relying party receives the string “2f6bc52c7527747eff263f4183c7f402”, it is completely opaque and reveals nothing about the identity of the user.

Working with known algorithms

As I mentioned before, Shibboleth is open source software. All of our code is publicly available, including the way that we calculate persistent identifiers. So how could a relying party use this knowledge to take the above opaque identifier and decipher the identity of the user it belongs to? Pretty simply, if they have a list of possible usernames, or can reasonably guess them. A list of usernames is far easier to get than you might imagine from unprotected university LDAP directories. Then you simply iterate through the list and run the same algorithm until you find the matching hash value. And even given a population of 30,000 usernames (the approximate number of students at USC where I worked), the following shell script churns through them in about 90 seconds:

~$ date && for i in `head -n 30000 /usr/share/dict/web2`; 
   do echo $i | md5 >/dev/null; done && date
Sun Aug  2 16:25:53 PDT 2009
Sun Aug  2 16:27:35 PDT 2009

To protect against such a brute-force attach to reveal the identity of a user, we added a secret salt to the hashing algorithm. That way, you can’t regenerate the hash without also knowing the salt value:

md5( "jsmith" + "http://idp.example.com/" + "http://sp.example.com/" 
    + "my-secret-salt" ) = cf79282c587897fb733d8338fe7bc9c2

It’s worth pointing out that, while use of a secret salt prevents a relying party from brute forcing the hash, it in no way prevents the identity provider from doing so, since they do of course know the secret salt. Though we never ended up needing to do this at USC, we built the tools that would enable us to identify the user a persistent identifier belonged to. In the event that a relying party reported that a particular user was abusing their system, we wanted to make sure we could identify who it was.

The realities of deployment

The above algorithm actually works pretty well for generating unique and secure opaque values for tuples of user, identity provider, and service provider. But what happens when one of those values change? At USC, users could request that their username be changed for a variety of reasons, and approximately 300 such requests were made each year. So using the above algorithm, a username change would change every one of the user’s generated persistent identifiers. And the reality is, businesses are often bought out by other businesses. What happens when “http://sp.example.com/” needs to change to “http://sp.new-company.com/”? That will effect the generated persistent identifier for every user. How do you deal with these realities? There are a number of ways, and I’ll outline just a few. There is no one right solution, as the policy and practices of a particular institution will greatly impact their decision of how to address these situations.

Perhaps the simplest option in some respects would be to simply not change which identifiers are used for generating the hash and instead map the new identifiers to the old values. Even if a user’s username has changed to “jjones”, you map it back to “jsmith” for the purposes of generated persistent identifiers. While this allows a deployment to continue using the same hashes, it does introduce the burden of maintaining this map of identifiers. And what is the institution’s policy with regards to re-issuing identifiers? If “jsmith” is allowed to be re-issued to a different user at some point in the future, that is going to create problems.

An alternate solution would be to simply use better identifiers. At USC, we had another user attribute called the “uscPVID” which we used instead of username. This was itself an opaque identifier that effectively served as the primary key within the enterprise directory. Even if all the other data for a user changed like their name and username, the uscPVID would remain the same. The same kind of persistent key can be obtained for relying parties by creating a lookup table that maps the relying party’s public ID to some more persistent internal ID. If and when a relying party changes their public ID, you simply modify, or add a new entry in your lookup table.

Finally, an identity provider could migrate from old to new identifiers, either with a one time update, or gradually. For a one time update, the identity provider would generate the old and new identifier for a user or set of users, and provide that information to the relying part out of band. The relying party would then be responsible for updating their database accordingly. Alternately, the new and old mapping could be provided gradually at the time of user login by using attributes. This was the technique used at USC for updating relying parties of changes to a different user attribute known as the USCID. For reasons I won’t bother explaining here, this 10 digit ID could change for a user when certain events occurred. To alert relying parties, we would include two attributes — the current USCID for the user, along with a list of “historical” USCIDs for that user. Relying parties were then responsible for updating any records they had for one of the historical USCIDs to the current USCID of the user.

Application to OpenID

The above hashing method could be used with little or no modification to create directed identity URLs for OpenID users. You could of course simply generate completely random IDs and store them in the database… that works too. In my next post however, I’ll be talking about a new kind of OpenID service I’ve been doing a lot of thinking about, an OpenID proxy which works to protect the privacy of OpenIDs that don’t support directed identity themselves. We’ll be using the above hashing algorithm, but doing some interesting things with the user identifier.

OpenID attributes

The OpenID Simple Registration Extension was designed, much like OpenID itself, to address a very specific use case — providing the most common data requested when registering at a new website (things like name, email, gender, etc). It defines a fixed list of nine attributes that can be provided, and it works well for what it does, but it’s a bit rigid when one tries to do anything more with it. Even before the final draft of the SREG extension had been published, work was already being done on a more flexible extension for attribute exchange. This new extension defined only a mechanism for transferring attributes, but didn’t prescribe a fixed list of attributes. But in order to be at all useful, two parties must be assured that they are both talking about the same attribute. Therefore an attribute registry was established at axschema.org and a number of attributes (formatted as URLs) have already been defined which will likely be useful in common scenarios.

Attributes in Higher Education

Developed in the early 90s at the University of Michigan, LDAP quickly grew favor with higher education institutions to replace heavier-weight directory protocols. A number of LDAP object classes were defined to provide attributes deemed useful in certain contexts, for example rfc2798 defined inetOrgPerson which addressed the needs of “Internet and Intranet directory service deployments”. Higher Education had its own unique needs, so in 2001 Internet2 published the first version of the eduPerson Object Class. If (and only if) institutions required additional attributes beyond those provided by some already published object class, it was common practice to define a local object class to house those attributes.

Within a few years, the SAML specification was published and Shibboleth began finding its way onto many college campuses. Though not required by SAML, Shibboleth recommended the use of URIs for SAML attribute names, and subsequently established a collection of URN values to represent common LDAP attributes, including those defined in the eduPerson object class. As before, if campuses needed to include additional attributes in a SAML assertion, they were defined within a local namespace. Today, it is very common to see a mix of attributes at USC including those defined by Internet2:

• urn:mace:dir:attribute-def:displayName
• urn:mace:dir:attribute-def:eduPersonAffiliation

as well as those we have defined locally at USC:

• urn:mace:usc.edu:gds:attribute-def:uscUSCID
• urn:mace:usc.edu:gds:attribute-def:uscStudentDegreeProgram

Applying Precedence

No one is pretending that the fixed set of attributes defined in rfc2426 is the definitive list of attributes that could possibly be useful in OpenID, no more than eduPerson was the exhaustive list of attributes for every university that used it. All that is being questioned is why did Attribute Exchange redefine all that was already established in vCard? I would strongly support Tantek’s third proposal of hosting the official hCard XMDP profile at http://microformats.org/profile/hcard, so that AX can leverage the hCard specification while maintaining URL based attribute names. And given that it appears as though no one is currently supporting AX, now is the time to make this change.

(For the non-technical, the title roughly reads “First try to reuse. Only if that doesn’t work, then reinvent.”)

So the question is, when OpenID is clearly a player in the future and part of that promise is about making things easier, more consistent and more citizen-centric, why would we go and introduce a whole new format for representing person-detail-data when a perfectly good one already exists and is so widely supported?

While I certainly agree with Chris that hCard has a role to play in the new world of online identity, I don’t want SREG or AX to be dismissed too quickly. I don’t believe he was suggesting that hCard would replace SREG/AX in all use cases, but I want to labor this point just a little bit so that others are not confused.

There is one huge benefit to SREG and AX that hCard doesn’t (and by it’s nature, can’t) address… releasing different attributes to different relying parties. This can take a number of forms. The one most commonly supported in OpenID providers today is the idea of personas - a complete set of attribute values that reflect your online identity within a particular context. You may have a “work” persona that includes your formal name, work email address and website, etc. Separately, you may have a “personal” persona that has some informal nickname along with a personal email address, website. etc. Separate still, you may have an “anonymous” persona that includes little (or fabricated) data about yourself. Depending on which relying party you are authenticating to, you may use a particular persona.

Varying this slightly, one can imagine a use case that will be become increasingly important as OpenID continues to expand. hCard is suitable for public attributes I want to post for the world to see, but what about not-so-public data? It will likely be some time before we start transferring credit card numbers over OpenID+AX, but I have no doubt it will happen eventually (as nervous as that makes even me). But even before then, I can imagine a host of data that I may need or want to provide to a relying party, but do not want to publish in my public hCard. As I included in my OpenID provider wish-list, OpenID providers will need to provide the tools to manage the policies controlling this release of data, a feature that has the potential to really differentiate one provider from the rest of the pack. This fine-grained level of control has always been a core requirement for the Shibboleth SAML Identity provider, and I would argue that the (currently beta) Shibboleth 2.0 IdP has one of the most powerful and flexible (and admittedly, verbose) filtering engines of its kind anywhere.

So there are really two issues at play in Chris’s question, that of the data format and separately, the mechanism for conveying that data. I haven’t really addressed the issue of the data format, because I agree with Chris in principal at least. His comparison table of attribute names shows a very clear overlap between the different methods, and perhaps it would be useful to work to consolidate some of that. But the mechanisms for making that data available address different, and equally valid, use cases and each play a role in making all of this stuff work.

Imagine being able to update your shipping address in one place when you move house and having all the online retailers you use receive the updated address immediately. Or changing your email address and having all the bugzilla instances you use pick up the new address instantly (perhaps requiring you to verify the new address first, of course).

While I agree this kind of experience would be very neat, it is not a use case for OpenID nor the Attribute Exchange extension. I am not surprised to hear someone say this, as this is a common point of confusion here at USC in regards to Shibboleth. When it comes to attribute delivery, both OpenID and SAML are primarily designed to provide data at the time of login ¹. So this means that if a user never logs in to your service, you never get any data about them. And if a user’s data changes since their last login, you don’t find out about until they next login, since that is the only time attribute delivery occurs. It is also important to note that you are typically only getting data about the user that is logging in, no one else. You can’t treat an OpenID provider as a lookup service to get data about some other user.

So how does a relying party update their data without waiting on the user to login again? One option is to have the data pushed to each service using some kind of messaging hub. There is a whole market for enterprise messaging complete with its own set of acronyms like ESB, EMS, EAI, and many many more. I’m particularly interested in a project developed by a colleague of mine in Sweden called JEvent, which uses the XMPP Publish-Subscribe protocol to deliver these types of messages using a standard XMPP server in the middle. Your Jabber messaging window becomes your console. :)

The other method of getting at user data is for each relying party to pull the data. This is the model we use at USC, where that mechanism for pulling data is LDAP. A number of services on campus use LDAP to address both use cases mentioned above — refreshing data about users, as well as looking up data about someone other than the user who logged in. So what’s the equivalent in the decentralized OpenID world? Well honestly there isn’t anything just yet but we have a good foundation. Thanks to microformats like hCard, you can publish your data in a standard, machine-parsable format that your online retailers or bugzilla instances could watch periodically. When you update your hCard on your homepage they can respond in kind, perhaps by immediately updating their databases or sending you a confirmation email of the change. Right now all of this data is decentralized, published on personal blogs and webpages. I don’t argue that this data should certainly be under individual control, but it puts an awful large burden on the individual service providers to do all that spidering. I don’t know what the answer is, but I’ve been thinking about an hCard crawler with an LDAP front-end as a simple place to start. Configure Apple AddressBook or Outlook to use LDAP, and you’re off and running.