PHP

First a quick note, to make sure this discussion does not get derailed. There is a time and a place to talk about these topics in the abstract. That is incredibly important work, especially in the development of these specifications, but that’s not what I’m currently interested in. I’m focused on developing solid PHP libraries to implement these technologies. Why PHP? Because that’s what WordPress uses, which is the current platform I’m targeting with the work I’m doing in DiSo. I know that PHP isn’t as sexy as Python or Ruby, but it’s what we’re using. I agree that we need solid libraries written in these other languages as well, but that’s not my focus. PHP is widely deployed and used, including companies very involved in implementing the Open Stack like Facebook and Plaxo (Luke, Joseph — I’m expecting some help from you guys :) ).

I’ll also note that I’m specifically targeting PHP 5. PHP 4 is no longer supported, and maintaining backwards compatibility (especially when talking about XML parsing) is a complete pain. This creates a problem with getting code into WordPress core, but I’m okay with that… they’ll move to PHP 5 eventually.

OpenID

Let’s start with the most mature library we’ve got. JanRain made a huge name for themselves in the OpenID community a couple of years ago by providing open source libraries in a number of different languages, including of course PHP. Like any library, there are a few weird things here and there, but by and large it is an excellent implementation that has served the community (including this developer) very well. Last week, JanRain announced that they were restructuring the development process of the PHP library to make it more open to developers. The code itself has moved from their internal darcs repository to github, they’ve added Luke Shepard of Facebook and myself as committers, and releases, bug tracking, etc will eventually be moved to the Google Code project. Going forward, we’ll be looking at trimming down the library a bit, removing support in core for older protocol versions and edge cases that weren’t really used, and overall making it easier for developers to use.

OAuth

There are two OAuth PHP libraries that I’m aware of, the “official” library stored in the OAuth Google Code project, and the Mediamatic library from Marc Worrell. The former library seems to have more users because of it’s exposure from the OAuth website, and is much lighter weight than the Mediamatic library (too much so for my taste). I initially chose the Mediamatic library for my work in getting OAuth working with WordPress, but eventually found some problems with the general library architecture. After some discussion with developers of both libraries, I’ve begun work on a new OAuth library. I re-architected the library from scratch, and then used a combination of the two libraries for much of the actual implementations. It’s probably about 80+ percent done, and should hopefully provide something both communities can work with.

Metadata Discovery

Discovery has certainly received the least amount of love from the development community, which is a bit ironic given that it’s a foundational part of almost every application of the Open Stack. There’s no shortage of metadata discovery and parsing libraries: Joseph Smarr contributed one to the xrds-simple Google Code repository, the OpenID library has its own, and the Mediamatic OAuth library has its own. Yet amazingly, none of these help you at all if you’re wanting to manipulate or publish a metadata document. They’re all half-baked, each written for a very specific use-case. What we need is a full implementation of the discovery protocols. And that, of course, is where it gets a little more complicated…

Disclaimer: If you really want everything there is to know about this subject, go read the writings of Eran Hammer-Lahav… I’m just going to gloss over it a bit.

Metadata discovery includes two steps: you need to know how to get the metadata about a resource, and you need to know what format that metadata is in so that you can parse it and make sense of it. OpenID uses a technology known as Yadis to retrieve the metadata document, which is in an XML language known as XRDS (Extensible Resource Descriptor Sequence). OAuth Discovery uses a combined and simplified version of these two known as XRDS-Simple. Discovery for OpenID and OAuth is more-or-less compatible.

Now, there is also work being done in the OASIS XRI TC (of which I’m a member) to develop the simpler, and more uniform successor to these protocols. Retrieval of the metadata will use a collection of methods known as LRDD (pronounced “lard”), while the metadata itself will be in a much simpler format known as XRD. While identical in spirit, these are complete rewrites of the previous specs. The new specs are not compatible with the old, but they are also designed so that they do not conflict either, so that both may be used simultaneously. Shifting to these new discovery protocols will certainly not be easy, but believe me when I tell you that it will be worth it. In fact, it’s absolutely essential for players like Google to implement OP-driven identifier selection (allowing users to login with OpenID by simply entering “gmail.com”).

So as I said earlier, we don’t have any real good discovery libraries for PHP. As part of my work on WordPress, I started development on a XRDS-Simple library in PHP. More recently, I created a separate branch of the code which implements LRDD+XRD exclusively. Realistically, we’ll probably need a library which handles both the old and new protocols for a while. The idea would be that none of the higher level libraries like OpenID or OAuth need worry about metadata discovery, except for maybe a lightweight wrapper around the discovery library. The new OAuth library I’m working on will do this from day one; the existing OpenID library will take a little while, but I think we’ll eventually see it rely on a separate library for discovery.

Feedback and Help

First of all, I welcome any feedback on the implementations that currently exist, especially the OAuth and discovery libraries I’m working on. They are not complete and most certainly not production ready, but they’re getting close. I’d also like to solicit development help, especially from people with larger deployments and/or a vested interest in this technology. All the new development is happening on github, so creating a clone to hack on is incredibly simple. Even if you don’t have development cycles you can put into this, I’ve already got at least one technical decision I need to make that I’d love feedback on, which I’ll be covering in my next post: “Why Does HTTP Suck So Much in PHP”.

• I have been listing will.norris.name as my homepage in my various social networks profiles and on blog comments. I’ve built up some Google page rank love through these links, and I want to make sure that is transferred over to willnorris.com. I also want any visitors that go to will.norris.name to be sent over to willnorris.com. The easiest (and correct) way to accomplish both of these is to setup a simple 301 “Moved Permanently” redirect.

• I have also been using will.norris.name as my primary OpenID to login to numerous websites. Many OpenID consumers will allow you to connect multiple OpenIDs or at least change your OpenID, so I can go through them all and update my account. But that will take a while, and I’d like to combine my domains now. The problem is that if I go ahead and setup a 301 redirect (as mentioned above), then it breaks my ability to use will.norris.name as an OpenID (see #4 under OpenID Normalization).

So I want Google and other visitors to see a permanent redirect to willnorris.com, but I don’t want to break my ability to use will.norris.name as an OpenID. I was originally planning to use Apache to perform the redirect, and I wasn’t sure if I’d actually be able to find a solution to my problem. Then I started thinking about WordPress request processing, and came up with the following bit of code:

add_action('wp', 'redirect_wp');

function redirect_wp($wp) {
    // only redirect plain home page requests
    if (!is_front_page() && !is_home()) return;
    if (!empty($_SERVER['QUERY_STRING'])) return;

    // don't redirect OpenID requests
    if (stripos($_SERVER['HTTP_ACCEPT'], 'application/xrds+xml') !== FALSE) return;
    if (stripos($_SERVER['HTTP_USER_AGENT'], 'openid') !== FALSE) return;
    if (empty($_SERVER['HTTP_USER_AGENT'])) return;

    wp_redirect('http://willnorris.com/', 301);
    exit;
}

So it’s pretty simple really… First, we only care about requests to the front page. Because I was only using will.norris.name as a one-page identity site, not a full blog, I only need to worry with requests to the front page. If I had been writing blog posts or other pages on this site, I would need to have different logic here. Second, we do some basic detection for OpenID requests — things like the content negotiation header for XRDS, or the “openid” string that appears in the user agent of JanRain OpenID libraries. If it doesn’t look like an OpenID request, we go ahead and redirect to willnorris.com.

Now of course this only works in my specific use case, but perhaps it will prove useful for others. For what it’s worth, the new work we’re doing on metadata discovery with XRD would prevent this problem, since we’re moving away from overloading normal HTTP requests where possible.

Update: I now also treat an empty user agent string as an OpenID request. This covers Blogger, which uses the OpenID4Java library. I’m fairly certain all major search engine spiders include a user agent, so this should be a fairly safe addition.

Additional Note for FastCGI Users

It’s worth noting as well that WordPress loses the HTTP status code when using FastCGI, as is used at Joyent. A comment in the code claims that it causes problems with some FastCGI setups, but I’ve not experienced that. This is very important, because Google will not transfer your page rank repuation if only using a 302 “Moved Temporarily” redirect; it must be a 301. A quick fix for this is to add the following to the very beginning of the above code:

add_filter('wp_redirect_status', 
    create_function('$s', 'status_header($s); return $s;'));

Early in the project, it didn’t seem like too many people were talking openly about the state of social networking. We all commonly referred to Facebook and MySpace as “silos” and “walled gardens”, but few people were talking about what the alternative would actually look like. Now, I’m sure there were far more people working on this than I realized… I have no doubt about that. But for me, it seemed like we were somewhat alone out there trying to figure out what protocols and technologies we could piece together to build a truly distributed social network. Because of this, I often described DiSo almost as a think-tank for developing the model and the technologies. We were writing code as well because we had to have a reference implementation that proved what we were talking about was possible, but I always de-emphasized that. Code comes and goes, dozens of implementations of these protocols will be written in different languages for different platforms, and that’s fine. What’s more important then, is the protocol itself. That’s what DiSo was all about… gathering (and creating when necessary) the collection of protocols necessary to make this stuff work. At least that’s how I viewed it, and explained it.

Today, the Open Stack is becoming very real. We have agreed upon standards for service and metadata discovery, authentication, API access, and contact information. We have commitments and production deployments from key players in this space including Google, Yahoo!, MySpace, AOL, Microsoft, and many others. What seemed like a small effort between a few individuals is now a full-scale shift of how we think about social interaction online. Just to be clear, I’m not trying to take credit for any of this stuff happening. The DiSo Project has played a small role in helping to shape and direct a few of the individual discussions, but this truly is a concerted effort between a lot of people who see the real potential for what we’re trying to do.

So where does this all leave DiSo today? Well, it’s obvious now that DiSo is not the think-tank for these technologies… they’re being developed all over the web, inside and between dozens of companies. Instead, I now put more emphasis on the code that we’re writing, because I think we represent a key principal of this entire Open Stack model.

The easiest example to give a layman for distributed social networking is “being able to interact with your Facebook friends using your MySpace account.” In the future, most people will likely have accounts with one or two of the large social networks or identity providers, and participate in the open web from there. With these large networks and providers at the table now, we can ensure that we develop a solution that will give users choice and the freedom to participate from anywhere. If this is truly going to be “user-centric”, then in fact, a user shouldn’t even have to join one of these large social networks in order to participate. Just like you can setup a server to host your own email instead of using a provider like GMail, you should be able to run your own server which provides the various pieces of the Open Stack. And that’s precisely where DiSo fits in — we’re working to provide the software that lets you participate in the open web from the comfort of your own domain. While there may be advantages to using one of the large providers, in order for this system to be truly open, then users must always have the option of maintaining complete control and running everything themselves. As soon as DiSo users become second-class citizens because they are not in one of the major social networks, then we’ve failed to achieve the level of openness we sought.

Today, all of my development for DiSo is being done in PHP, and most of it specifically for WordPress. Between myself, Steve Ivy, and Stephen Weber, we have basic WordPress plugins for OpenID, OAuth, XRDS-Simple, rich profiles (hCard), Activity Streams, contacts, and permissions. Many of them have work yet to be done before they are really stable and ready for mass consumption, but many can be seen right here on willnorris.com. Six Apart is also developing implementations on top of Movable Type, including the recently announced Motion, which provides some real cool functionality around activity streams.

Here Be Dragons!

Let me say first and foremost, don’t use this on a production blog. I always say that when I blog about unreleased code, but this time it’s much more important. There are major database changes in this version… changes which are non-trivial to reverse. There is a very good chance there will be more database changes before the final release, and there will not be an upgrade path from this development version (there will however be an upgrade path from the last stable version… 2.2.x).

What’s New

For now, I’m just going to have two follow-up posts talking about changes in the coming release. I’m sure I’ll overlook something and may have to add a third post, but for now we’re looking at:

• Making the plugin more stable, extensible, and overall simpler
• OpenID Providing and Delegation

Test it Out

The current plugin can be checked out from the DiSo subversion repository; grab the 3.0 branch . In addition, it requires a special branch of the XRDS-Simple plugin to provide all the XRDS publishing stuff. If you have a previous version of wp-openid installed, you must deactivate and reactivate the plugin for the database changes to be applied. Again, keep in mind that these will be somewhat substantial changes to the OpenID portions of the database, so don’t do this on your production blog just yet. Please direct any support questions to the DiSo Mailing List.