So what am I to do? The most obvious answer is to simply use one of the larger hosted OpenID providers that support such a feature, like Google. But I have some real problems with that, both philosophically and practically. Fortunately, I think I may also have a better solution.

Maintaining Control

My first knee-jerk reaction to being forced to use an OpenID provider like Google is a purely philosophical argument. If the whole point of OpenID was to create a completely distributed identity ecosystem, then isn’t this a huge step backwards if we’re no longer allowing for self-hosted OpenIDs in certain cases? While I think it’s an unfortunate situation, it is a completely understandable one. There really is no good way to maintain the privacy of the user, as I demonstrated above. Remember that OpenID, unlike SAML, was never designed to mask the identity of the user… quite the opposite, in fact! When you take any technology and try to apply it in situations it was not originally intended for, you have to be prepared for the reality that it’s not going to fit quite as nicely as one might like.

So philosophical arguments aside (as valid as they may be), what other reasons might one have for not wanting to use a third-party OpenID provider? One of the biggest in my mind is privacy. Have you ever looked at the “Visited Sites” page on MyOpenID or a similar provider? It shows which sites you’ve logged into, how many times, when the last login was, etc. Pretty neat data to have at your disposal as a user, right? But also potentially pretty scary data for JanRain to have for every user. Now, I’m not suggesting that JanRain or any other OpenID provider purge this data… it’s actually a really valuable feature and I think it’s in the user’s best interest to have access to it. But let’s remember what started this conversation… directed identity.

While directed identity can be used for everyday logins, it really shines when maintaining user privacy becomes very important. What if I’m a whistleblower who wants to report unsafe business practices to the government? What if I participate in some taboo subject matter online that I want to keep hidden from family and friends. What if I am a political dissident, and revelation of my identity could result in imprisonment or even death? What if I simply want to exercise my constitutional right to privacy? While some of these are rather exotic examples, they illustrate my point pretty well. There are cases where privacy is truly important. As I demonstrated in my last post on directed identity implementations, there are algorithms that can adequately protect the identity of the user from the site they are logging in to. But remember, those algorithms do absolutely nothing for protecting my identity from my identity provider. If privacy is my goal, then why would I trust Google or JanRain with my daily activities any more than anyone else?

In Andy Oram’s post on this topic “Shortening cookies: Using OpenID to improve government privacy online” and his follow-up “Privacy and open government: conversations with EPIC and others about OpenID”, he talks about the possibility of the federal government running an OpenID provider which is used to authenticate users to other government websites and services. First of all, I don’t think the government needs to stand up yet another OpenID provider. The private sector has done a pretty good job so far of making sure people have portable identifiers, whether they actually realize it or not. But more importantly, I simply don’t trust the government to maintain and respect my privacy. In discussing whether the government is actually the right party to run this service, Oram mentions:

The problem is whether visitors can trust any particular server 1) to stay up, 2) not to go out of business, 3) not to leak information, 4) not to abuse the information for private gain, and 5) not to cave in to government pressure and release information outside of the scope of the law.

It’s really the last point that bothers me. This is a system that, by design, holds the links between the private identities of citizens, and the anonymous identifiers they are using to communicate with the government. Not only are these individuals potentially screwed if the system is broken into, but also if some judge decides there is justifiable cause for revealing these identities. Privacy is a matter of policy, rather than technical design. This is not sufficient. I think we need a system that protects user privacy as a technical feature, not simply as a policy decision. We need a system that couldn’t reveal the identity of a user even if the Chief Justice himself were to order it.

OpenID Proxy

My idea for such a system is actually very simple in its design. At a high level, it’s not unlike Emailtoid, JanRain’s RPX, or the never really launched Vidoop Connect. It is an OpenID proxy that stands between the actual relying party and OpenID provider, so that the two never actually communicate directly to one another. The proxy faces both parties and therefore implements both sides of OpenID — it provides an OpenID Provider implementing directed identity which communicates with the actual relying party, and it also implements an OpenID relying party which communicates with the actual OpenID provider. Let’s look at the basic user flow…

Our user, Alice, goes to OSHA.gov to report the unsafe work environments at her job. Fearing retribution from her employer if they were to find out she reported them, Alice wants to remain anonymous in her communication with OSHA. When logging in to OSHA’s whistleblower site, Alice enters the URL of an OpenID proxy, “proxy.example.net”. This proxy could be run by the government itself, a private citizen’s rights organization, it doesn’t really matter. OSHA’s OpenID relying party communicates with the OpenID Proxy’s server, and begins an identifier select OpenID flow, which results in Alice being sent over to the proxy. Now, instead of having to create a new account at the proxy, Alice uses her real OpenID URL, “alice.example.org” to login. After successfully authenticating, the OpenID proxy uses a hashing algorithm to generate an opaque URL for Alice, and returns that identifier back to OSHA. So Alice was able to use her own self-hosted OpenID URL in a privacy preserving manner — OSHA has no way of identifying her based on the resulting directed identity issued by the proxy.

But what about the proxy itself? As we’ve already mentioned, the OpenID provider which generates the directed identity is able to determine the user the identifier belongs to, even if a secret salt was used as part of the hashing algorithm. In order to protect Alice’s identity from even the OpenID proxy, we need to do two more things. Let’s look again at our hashing algorithm for generating directed identities:

md5( username + openid_provider + relying_party + secret_salt )

For OpenID, that username would simply be Alice’s OpenID, “alice.example.org”. If subpoenaed, the OpenID proxy would be able to determine whether a given directed identity indeed belonged to “alice.example.org” simply be running it through the above algorithm. Remember that the secret salt is what prevents the relying party from using this same technique to identify the user? Well what if we add a little secret salt of our own to the username portion of the algorithm? What if we replaced “username” with “username + user_salt”? That way, even the proxy itself wouldn’t be able to replay the hashing algorithm without actually knowing the correct salt value to plug in there.

Now, before I talk about the user salt itself, there is one caveat that must be pointed out… this is the second of the “two more things” I mentioned above. The user salt is not exactly secret, because it must be provided to the OpenID Proxy so that it can be used to generate the directed identifier. The thing that protects user privacy is that the proxy must not record the value of the user salt or the user OpenID. It can’t log the values anywhere and it can’t store them in a database. It must use the OpenID and salt to generate the directed identifier, and then get rid of them. Otherwise, it would still be technically possible to trace the identifier back to the actual user. This would be one reason why it might make sense for such a system to be run by a citizen’s rights organization that is generally more trusted to do everything possible to protect user privacy. It’s also worth noting that nothing in this architecture suggests that there would be only one OpenID proxy… there can, and should, be several proxies for user’s to choose from, all using these same (or similar) techniques.

Adding user salt

There are two methods I can think of to add the user salt mentioned above. The first is certainly the most straight-forward, but also a little more difficult for some users. If Alice’s OpenID provider were to itself return a directed identity to the OpenID proxy, that would be sufficient for salting the proxy’s hashing algorithm. So long as that directed identity is not being stored by the proxy, there would be no way to trace back any identifiers the proxy generates. It’s worth noting that in this case, a self-hosted directed identity actually would be sufficient. Remember earlier when I mentioned that the URL “http://willnorris.com/openid/9eb4d59c1d488a4” wouldn’t do much for protecting my identity? That is true, but for the purposes of salting the proxy’s hashing algorithm, it would work just fine. But this of course still requires that the user have access to an OpenID provider that supports directed identity. The WordPress OpenID plugin which I use does not currently, so this wouldn’t work for me.

The second method of providing a user salt requires a slight modification to the OpenID provider, but should be a bit easier to do. The salt could be provided separately by the OpenID provider as an extension on the OpenID transaction itself. To make things easier, it could simply be a new Attribute Exchange attribute. No need for new OpenID extensions, just a new attribute which represents a “user salt”. The OpenID proxy would then combine the user’s OpenID together with the salt value, and use that to generate the final directed identity that is returned to the relying party. If the proxy were subpoenaed to verify if a given directed identifier belonged to “alice.example.org”, it would be unable to do so without also knowing the user salt. And as long as the salt was not being recorded anywhere, the proxy would be completely incapable of verifying the user that an opaque identifier belonged to.

So what about using some other things as the user salt? How about the association used in the OpenID transaction? Remember that the point is still for Alice to be able to login to OSHA.gov over time and be recognized as the same (possibly anonymous) user. In order for this to work, she must return with the same directed identifier from the OpenID proxy each time. And in order for the proxy to ensure that it generates the same directed identifier, it must use the same input values. While an OpenID association is intended to be relatively long-lived, it can, and often does, change over time. When this happens, the proxy would end up generating a new directed identifier for Alice, and she would no longer appear as the same user at OSHA.gov.

Some practical matters

So there are some additional practical matters that I think must be considered for this solution to truly work. First of all, what happens if the proxy’s secret salt is compromised? This would certainly be a bad thing, but not quite as bad as it would be in a standard directed identity situation. Remember in our final algorithm from last time, the secret salt was the one and only key that protected the privacy of the user in a generated directed identifier. If the salt is compromised by a party, they could then brute-force the hashing algorithm and identify the true identity of a user associated with a directed identifier. But remember that one of the goals of the proxy is to protect the true identity of the user from even the proxy itself… that’s why we added the user salt. So even if the secret salt of the proxy were compromised, an attacker would be unable to identify the user associated with a given directed identifier. Additionally, if the proxy were aware of the compromised salt, they could transition over to a new secret salt using the same method I mentioned last time that USC uses to update relying parties of new USCID values. The proxy would simply inform the relying party that user X, identified by this opaque identifier, was previously identified by this other opaque identifier, and their records should be updated. Nothing about the user herself is revealed, only the transition from one opaque identifier to another.

What if the user still wants to maintain different personas for different relying parties? For example, what if Alice wants to reveal the name of her employer via an AX attribute when communicating with OSHA.gov, but not when communicating with WhiteHouse.gov? Technically, the solution is pretty simple, but it’s not as user friendly as I would like. The OpenID proxy would simply need to embed the trust root of the actual relying party inside its own trust root that it presents to Alice’s OpenID Provider. Part of the OpenID flow is that a provider prompts the user if they trust the site that is asking them to authenticate, and additionally if they want to release any additional attributes to the party. The relying party is identified using a URL known as the “trust root”. So a user may see a prompt (borrowing from MyOpenID’s language) along the lines of:

You are signing in to proxy.example.net/ as http://alice.example.org/.

Traditionally, the decision that Alice makes here is recorded so that she is not prompted each time she logs in. So if she wants to make a different decision depending on which site she is logging in to, then the OpenID proxy would need to change its trust root. One example might result in a prompt that looks like:

You are signing in to proxy.example.net/site/osha.gov as http://alice.example.org/.

This would record her decision specifically for “OSHA.gov via proxy.example.net”. Using this method, she could easily release different attributes for different relying parties, without any changes to her OpenID provider whatsoever. While I don’t like the idea of the user having to parse apart that URL to figure out what it means, it’s at least workable. As a long term solution, a new OpenID extension could be defined which provides a more human friendly version of the trust root which explains to the user more clearly where her data is going. In fact, I think that has already been proposed and being worked on.

Proof of concept

Though the explanation of the concept ends up being a little wordy, the design itself is actually quite simple. Because of the strict requirement that identifiers not be stored, there is actually not much to it. The only database storage would be for OpenID associations and nonces… everything else is calculated in real time. I’ll be working with Michael Richardson this week to be build a working implementation to show how it would work. I’m also beginning conversations with EPIC to help advise them on their recommendations to OMB regarding federal website cookie policies. I welcome any questions or comments on this approach to privacy with OpenID. Some of the specifics I just really thought about this last weekend, so I may be overlooking a few things. But the overall architecture is nothing that hasn’t been done before, and has a strong, proven track record.

Shibboleth and SAML

Shibboleth is an open source web single sign-on product which is very popular with universities around the world. It is primarily an implementation of the Security Assertion Markup Language (SAML), but supports other identity protocols as well. SAML v2 defines a specific type of identifier that may be used to identify a user within a transaction known as a Persistent Identifier. The name itself does not convey it’s full meaning, but it is defined as “a persistent opaque identifier for a principal that is specific to an identity provider and a service provider or affiliation of service providers.” This is SAML’s directed identity.

Shibboleth’s implementation of persistent identifiers has matured a bit over the years, but the main algorithm remains the same. At it’s simplest, the identifier is simply a hash of three things: an identifier for the user, an identifier for the identity provider, and an identifier for the relying party. Using md5 as our hashing algorithm, and using a few made up values, this would look like:

md5( "jsmith" + "http://idp.example.com/" + "http://sp.example.com/" ) = 
    2f6bc52c7527747eff263f4183c7f402

When a relying party receives the string “2f6bc52c7527747eff263f4183c7f402”, it is completely opaque and reveals nothing about the identity of the user.

Working with known algorithms

As I mentioned before, Shibboleth is open source software. All of our code is publicly available, including the way that we calculate persistent identifiers. So how could a relying party use this knowledge to take the above opaque identifier and decipher the identity of the user it belongs to? Pretty simply, if they have a list of possible usernames, or can reasonably guess them. A list of usernames is far easier to get than you might imagine from unprotected university LDAP directories. Then you simply iterate through the list and run the same algorithm until you find the matching hash value. And even given a population of 30,000 usernames (the approximate number of students at USC where I worked), the following shell script churns through them in about 90 seconds:

~$ date && for i in `head -n 30000 /usr/share/dict/web2`; 
   do echo $i | md5 >/dev/null; done && date
Sun Aug  2 16:25:53 PDT 2009
Sun Aug  2 16:27:35 PDT 2009

To protect against such a brute-force attach to reveal the identity of a user, we added a secret salt to the hashing algorithm. That way, you can’t regenerate the hash without also knowing the salt value:

md5( "jsmith" + "http://idp.example.com/" + "http://sp.example.com/" 
    + "my-secret-salt" ) = cf79282c587897fb733d8338fe7bc9c2

It’s worth pointing out that, while use of a secret salt prevents a relying party from brute forcing the hash, it in no way prevents the identity provider from doing so, since they do of course know the secret salt. Though we never ended up needing to do this at USC, we built the tools that would enable us to identify the user a persistent identifier belonged to. In the event that a relying party reported that a particular user was abusing their system, we wanted to make sure we could identify who it was.

The realities of deployment

The above algorithm actually works pretty well for generating unique and secure opaque values for tuples of user, identity provider, and service provider. But what happens when one of those values change? At USC, users could request that their username be changed for a variety of reasons, and approximately 300 such requests were made each year. So using the above algorithm, a username change would change every one of the user’s generated persistent identifiers. And the reality is, businesses are often bought out by other businesses. What happens when “http://sp.example.com/” needs to change to “http://sp.new-company.com/”? That will effect the generated persistent identifier for every user. How do you deal with these realities? There are a number of ways, and I’ll outline just a few. There is no one right solution, as the policy and practices of a particular institution will greatly impact their decision of how to address these situations.

Perhaps the simplest option in some respects would be to simply not change which identifiers are used for generating the hash and instead map the new identifiers to the old values. Even if a user’s username has changed to “jjones”, you map it back to “jsmith” for the purposes of generated persistent identifiers. While this allows a deployment to continue using the same hashes, it does introduce the burden of maintaining this map of identifiers. And what is the institution’s policy with regards to re-issuing identifiers? If “jsmith” is allowed to be re-issued to a different user at some point in the future, that is going to create problems.

An alternate solution would be to simply use better identifiers. At USC, we had another user attribute called the “uscPVID” which we used instead of username. This was itself an opaque identifier that effectively served as the primary key within the enterprise directory. Even if all the other data for a user changed like their name and username, the uscPVID would remain the same. The same kind of persistent key can be obtained for relying parties by creating a lookup table that maps the relying party’s public ID to some more persistent internal ID. If and when a relying party changes their public ID, you simply modify, or add a new entry in your lookup table.

Finally, an identity provider could migrate from old to new identifiers, either with a one time update, or gradually. For a one time update, the identity provider would generate the old and new identifier for a user or set of users, and provide that information to the relying part out of band. The relying party would then be responsible for updating their database accordingly. Alternately, the new and old mapping could be provided gradually at the time of user login by using attributes. This was the technique used at USC for updating relying parties of changes to a different user attribute known as the USCID. For reasons I won’t bother explaining here, this 10 digit ID could change for a user when certain events occurred. To alert relying parties, we would include two attributes — the current USCID for the user, along with a list of “historical” USCIDs for that user. Relying parties were then responsible for updating any records they had for one of the historical USCIDs to the current USCID of the user.

Application to OpenID

The above hashing method could be used with little or no modification to create directed identity URLs for OpenID users. You could of course simply generate completely random IDs and store them in the database… that works too. In my next post however, I’ll be talking about a new kind of OpenID service I’ve been doing a lot of thinking about, an OpenID proxy which works to protect the privacy of OpenIDs that don’t support directed identity themselves. We’ll be using the above hashing algorithm, but doing some interesting things with the user identifier.

OpenID History

It’s important to understand a little bit of the history of OpenID. The technology was created in 2005 by Brad Fitzpatrick while working at Six Apart as a means for bloggers to leave authenticated comments on each others blogs without having to create new accounts. Since the target community was bloggers, everyone already had a blog URL that they identified with, so that made for a convenient portable, identifying, and globally unique identifier for users. In slightly more technical terms, we would refer to these identifiers as “omnidirectional” and “readable”.

The readability of an identifier is exactly what you would think, and can be determined quite easily. Does the identifier itself give any clues as to the object it identifies? Blog URLs were intentionally chosen as identifiers because they were easily recognizable. A non human-readable identifier is generally referred to as being opaque. You are not able to discern anything about the resource simply by looking at its identifier. Take a UPC barcode for example — while it uniquely identifies a product in a store, the number itself is meaningless without looking it up in a database.

The second important property of an OpenID is in being omnidirectional. The direction of an identifier really just refers to the contexts in which that identifier is used. One of the original goals of OpenID was to have a single portable identifier that users could use on many different sites across the web. By contrast, a “directed” identifier is one that can only be used in certain contexts (generally, just one).

New Use Cases

Over time, the use cases in which OpenID was applied expanded, and the technology was forced to mature. Despite deliberate attempts NOT to include profile data in the original protocol, a new extension was soon created that would allow basic registration data like name and email address to be passed along on top of OpenID, revealing additional data about the user. Just as people are not one dimensional in real life, it was quickly apparent that there was value in allowing users to maintain multiple sets of identity data, generally called “personas”, that could be presented to different websites. I could have a “personal” persona which included different data than my “work” persona. If I really wanted to keep these parts of my life separated, I could even use different OpenIDs for different sites.

I think you could say that this is where many of the current usability problems with OpenID began. As users were having to manage multiple identities, and sometimes not remembering their URL at all, a new mechanism was devised to make things easier for users. Instead of typing in their full OpenID URL at a consumer site, the user could simply enter the URL of their OpenID Provider. This would allow the consumer site to start the OpenID authentication flow and send the user over to the right OpenID provider. The user could then authenticate to the provider, select a particular OpenID URL and persona if they have multiple, and the correct OpenID and data would be returned to the consumer site. This new flow that was included in OpenID 2.0 was never really given a good name, but is referred to in the spec as “OpenID Provider driven identifier selection”. Just rolls right off the tongue doesn’t it? This is why almost no one calls it by the right name, it’s a mouthful. But it is at least accurate — the OpenID Provider is responsible for having the user select the appropriate identifier for that particular transaction.

OpenID and Privacy

But there was another, perhaps more pressing, use-case that led to the development of the “identifier select” flow. While I as a user can maintain different personas which I use at different sites, what if I want to remain completely anonymous? What if I don’t want to reveal anything about myself, yet still be recognized as the same user each time I login to a particular site with my OpenID? Have you ever used Yahoo!’s OpenID provider by simply typing “yahoo.com” into an OpenID field? If you have, you may have noticed that Yahoo! gives you a choice of what OpenID URL you want to use, including one that is completely opaque (remember talking about “readibility” earlier). I am given two choices when I login:

• http://www.flickr.com/photos/wnorris
• https://me.yahoo.com/a/YN.TrVBnuIAvmAk7teEzbLW_MQ-

I can use my Flickr URL which links to my photo stream, and subsequently lots of other information about me, or I can use this opaque URL that reveals nothing about me. If I were to login to an OpenID enabled site using the second URL, there would be no way to identify which Yahoo! user it belongs to, or anything else about me — it is completely opaque. Well… except for that fact that I’ve just publicly revealed what my “secret” Yahoo! OpenID URL is. This means that anywhere I have previously used this URL can now be linked back to me. Not to worry, I’m pretty sure I’ve never used it anywhere except for testing.

But even without me revealing what my URL is, you could begin to build a profile of me. Without knowing anything else but my OpenID, you could search for that URL on various websites and piece together different things I may have said or done. Maybe I mentioned my city on one site, and part of my name on another. The more I use that “secret” OpenID, the more I reveal about myself, and the less “secret” it becomes. Even if my data on these sites was not publicly accessible, what if multiple site owners where to start building such a profile about me behind the scenes without me knowing? What if multiple government websites were to build such a profile about me? This is known as collusion, and is precisely what our next and final measure is intended to protect against.

Directed Identity

As we’ve seen above, OpenID Provider driven identifier selection does not necessarily imply anything about the readability of the subsequent URL that is returned. We’ve also seen that just because an identifier is opaque does not necessarily guarantee our privacy online. The final piece of our puzzle takes us right back to where this discussion started — directed identity. If you recall, the “direction” of an identifier refers to the context in which it is used. And a directed identifier is one that is typically used within a single context, or with a single party. So when we talk about “directed identity” in terms of OpenID, we mean that a different OpenID URL is used for every website you login to. While this does not necessarily mean that the identifier is also opaque, it’s pretty useless if it isn’t.

This concept isn’t new to identity or to OpenID. A very early identity company Sxip provided exactly this feature:

In our implementations of a Homesite, we let the user select which persona they want to be at a new site. One of those is an “anonymous” persona that will have a unique URL for each site.

This lets the user decide on a site by site basis what is disclosed.

— Dick Hardt

To my knowledge, the only OpenID provider that implements true directed identity today is Google (and Sxip still, I assume). If there are others I’m not aware of, please leave a comment and let me know. Remember that Yahoo! doesn’t implement directed identity because, even though they use “identifier select” along with an opaque identifier, that identifier is not unique for each website you login to.

Recap and Why this Matters

So to recap the three basic terms, and the way in which they build upon each other:

• OpenID Provider driven identifier selection (or identifier select for short) refers to the ability for a user to enter the URL of their OpenID Provider into an OpenID field rather than their personal OpenID URL. This is a feature of OpenID 2.0, and will result in an actual user OpenID URL being returned to the consuming site. This says nothing about the nature of that URL, and can be implemented simply as a user convenience.

• An opaque URL is one that does not itself reveal any information about the user it identifies. Any practical use of opaque identifiers necessitates the use of identifier select, since it is not realistic to have a user remember and enter a long and meaningless OpenID URL. Opaque identifiers protect user privacy.

• A directed identifier is an opaque identifier which is unique for a given site. The same OpenID URL is continually returned to a given consuming site, but no two consuming sites are ever given the same OpenID URL for a user. Directed identity protects against collusion.

Today, identity data is being thrown around pretty loosely without much regard to how it is being used, but that is quickly changing. Slowly but surely, we are seeing reputable companies involved in high value transactions express interest in what federated identity can offer. Remember how much buzz the administration of then President Elect Obama created when they implemented OpenID via Intense Debate on Change.gov? Just look at last week’s announcement from the White House regarding HTTP cookie policies on federal websites. Now tell me they are not going to be looking at directed identity if and when they were ever to implement federated identity for real. On that day, it will become very important that the community (and especially OpenID Providers) understand the difference between “identifier select” and “directed identity” if they want to play ball with the government.