So I’m off to San Francisco tomorrow morning, where I’ll be presenting AllPeers at the Web 2.0 conference’s Launch Pad event (scroll to bottom of page) next Wednesday. If anyone is going to be there, stop by and say hi! If you can’t make it but want to meet up, send me a mail. I’ll be running around quite a bit, but hopefully I’ll have time to post some more about AllPeers over the course of the week, as well as commenting on what I see at the conference.
Once upon a time I had a shortish spell as a Java programmer, using the outstanding Eclipse IDE. One of the things I liked most about it was the JUnit integration. Our project (or rather our customer’s project) was very large and complex, so we had a whole suite of tests that we would run before checking in code. The tests were displayed in a sidebar with a progress bar at the top, which turned red if any tests failed. Each individual test was flagged as well if it failed.
Anyway, I decided that we needed the same thing for AllPeers, but hosted inside Firefox. I couldn’t find anything like it, however, so I had to scratch the itch myself. The result is FoxUnit, my modest attempt to imitate what I liked about Eclipse/JUnit but tailored to the Firefox environment. If a much better implementation of this exists that I didn’t discover, don’t even tell me about it. I’d just start crying.
Disclaimers:
- This is a first version, and I focused on making it do what we need, not what any given Firefox developer might possibly need. So it’s pretty limited and doubtless laden with bugs. If it hurts you, your machine or someone dear to you, I deny all responsibility. I’m hoping it will evolve over time into something more functional and robust.
- It only runs on Deer Park. I have no idea if it would run on 1.0 as I was too lazy to test it. If someone wants to try, I certainly won’t object.
- The setup/teardown stuff doesn’t work that well. I need to fix that soon. Use them at your own risk.
- The progress bar doesn’t always do what you’d expect, especially when you’re not running all the tests. Also needs to be fixed.
- You can’t debug scripts in the test suite. I believe (on scant evidence) that it would be possible to make this work, but it’s probably not a trivial effort. Something I plan to do one day.
- The code itself is kind of spaghettish. I’m planning to reorganize it when I have time.
The basic idea is that you have a project file, which lists all the modules that you want to test. I’ve put a simple sample project file here. Each module is contained in an XML file; once again, there’s an example here. Modules are made up of tests, which are made up of test cases. Only the latter contain actual code, which is Javascript embedded in the XML file.
When you load the project, the entire hierarchy (Project -> Module -> Test -> Test Case) is displayed in the FoxUnit sidebar. Click “Open Project” and browse to your project XML file. I strongly recommend that you download the sample files (make sure they are in the same directory) and try them out before attempting to write your own. Press “Run Tests” to run everything, or right click on any node to run just the tests under that node (i.e. all tests in a module, all test cases in a test, or even just one specific test case). When a test case fails, it gets labeled with the red “FAIL” notice, as do all of its parent nodes, so you can see which modules or tests contain failed test cases even if they are collapsed in the tree view.
Download FoxUnit here
Run it with View/Sidebar/FoxUnit or just press Ctrl-Shift-U.
In the conclusion to my very first Peer Pressure essay, written about a year ago, I speculated about what might be killer features for a next-generation web browser. Looking at this list, it’s clear that, while progress has been made on a couple of fronts, there’s still a long way to go to achieve the vision of a “Web 2.0″ browser. Identity management is a burning hot topic, but not much has happened yet in terms of real-world implementations for actual users. Editing capabilities are mostly in the same shambolic state they were twelve months ago (though, thanks to rich web browser-hosted editors like Flock, there’s some hope on the horizon). Peer-to-peer communication is still divorced from the web browsing experience. And browsers continue to speak HTML fluently but develop a thick foreign accent when forced to communicate in XML.
We’ve focused heavily on some of these areas in developing AllPeers 2.0, and some new ones have cropped up along the way. I’ve already talked about our RDF/XML capabilities, which are key to achieving data reusability. I’ll try to discuss the rest in more detail over the next few days, starting with identity.
Doc Searls has some interesting commentary on his blog about the need for decentralized identity infrastructure. I particularly like his assertion that a workable identity infrastructure will be bottom up rather than top down. To achieve this vision, however, you need to have some sort of decentralized, extensible framework.
Our approach to identity is to use a basic profile that contains almost no information: just your name and a unique ID. This profile takes the form of an RDF resource, so it’s easy to extend it by linking it to further resources. Each resource can be signed by a third-party to verify authenticity. Essentially, we’re replacing the built-in Firefox password manager with an analogous system based on digital certificates that is far more powerful and secure. This differs from systems like Microsoft’s InfoCards mainly in its braindead simplicity. We haven’t invented anything; we’re just serving up signed RDF resources on demand.
Strong identity on the client doesn’t just mean single sign on. It also means that you can build up a very rich profile on your own machine, instead of having bits and pieces spread across a thousand web servers (run, of course, by evil faceless corporations). This, in turn, opens up the possibility of plugging-and-playing remote services to produce cool new applications (imagine bolting matchmaking onto a massively multiplayer game, for example). We’re not expecting our system to take over the world, considering the massive competition, but hopefully it will serve us well until a clear winner emerges in the identity stakes.
[web 2.0]
A propos of nothing: The New Yorker’s take on intelligent design. It’s hilarious.
Comments Off
Slashdot announced yesterday that they have finally overhauled the site’s HTML code. This is undoubtedly long overdue, but it does mean that my Greasemonkey script Slashdot Live Comment Tree no longer works, since it’s tied to the old HTML format.
I’m afraid that I really don’t have time to update the script right now. We’ve in crunch mode with AllPeers, and I’m flying to the States at the end of next week for the Web 2.0 Conference (where I’ll be presenting AllPeers at the LaunchPad session on Wednesday).
It’d be tremendously cool if someone could take a crack at this. I get the impression that the script is used by a fair number of people (it’s been downloaded a few thousand times), so this is your chance to improve the life of hoardes of Slashdotters, see your name in lights, hear the adoring cheers of the thronging masses… or, at very least, to earn my heartfelt gratitude.
While the initial spark of inspiration behind AllPeers centered around the use of peer-to-peer network topologies, I actually think that the biggest innovation is in the way that we structure and manage data. I’ve been active in the XML space for almost a decade, since I first heard the term at an SGML conference in Chicago. Yet despite a fantastic technical specification, tremendous buzz and widespread usage of XML as a serialization format, the standard has yet to live up the the vision of its creator, Jon Bosak. Jon’s idea was that XML would “give Java something to do”, providing structured data instead of mucky HTML that smart client-side apps could munch on. Java may not be the white-hot property that it once was, but the general idea is still as brilliant as it was back in 1997.
The reason for this failure is that XML itself doesn’t provide enough of the complete solution. RDF is in many ways a step in the right direction, filling in the gaps so that global interoperability of structured data can be achieved. But RDF has suffered from excessive complexity and a lack of effective tools for building fully functional applications. The missing pieces, in my view, include:
- Mechanisms for update of replicated resources.
- Efficient client-side storage and retrieval.
- Frameworks for generating attractive user interfaces.
- APIs that any sane developer would actually want to use.
- An approachable system for designing and distributing schemas for describing RDF data structures.
Now I’m not going to sit here and claim that we’re going to singlehandedly make the semantic web a reality. But I do think that AllPeers could have a real impact. In reality, P2P is as much about pushing processing onto the client as it is about making efficient use of network resources. This is essential if an application is to make use of structured data, and it’s especially relevant because we’re hosted inside Firefox. Firefox has a built-in RDF infrastructure, but it takes a lot of effort to do anything useful with the data. You could say that our goal is to make it as easy and intuitive to browse RDF data as it is to surf HTML pages.
AllPeers includes a very efficient storage module for storing and retrieving RDF resources in an SQL database (SQLite, to be specific), so you can process the large volumes of data that are characteristic of real-world applications. We track resource distribution and update replicated resources automatically when the original version changes. We use Relax NG schemas to describe resource formats and associated metadata, so you can enforce validity and organize metadata about these formats in a simple, structured way rather than in messy, fragile code.
Firefox already has a decent framework for creating user interfaces based on RDF, and it’s slated to get a whole lot better. So about the only thing that’s still missing is a less verbose API for non-deities to program to. I have some ideas about this as well. Just imagine an E4X API that hooks automatically into the abstract XML serialization of an RDF resource. That probably sounds like goobledy-gook, but the net net is that it could make RDF programming accessible to a much broader segment of the developer population.
Update: I just noticed that XML.com is running a rather eccentric article on “the difference between XML and RDF”. Enlightenment is a mere click away… or is it?
[web 2.0 rdf xml]
The AllPeers saga began in October 2002 when I visited Cedric (now our CEO) at his home in Aix-en-Provence. Instead of doing whatever it is that normal people do when on vacation (lying comatose on the beach, most likely) we spent an inordinate amount of time talking about Ced’s idea for a peer-to-peer community. Having run a couple of large online communities, he had experienced firsthand the cost (in servers and bandwidth) of supporting a growing user base with centralized infrastructure. He theorized that moving to a peer-to-peer architecture could result in significant cost savings and thus represent a real competitive advantage.
After mulling this over for a while, we started to work on a platform for creating peer-to-peer applications. The platform approach is particularly compelling in this context because it’s so hard to develop this type of application. By nailing the difficult bits ourselves, we aimed to make development of peer-to-peer software quicker and more accessible to a wider range of developers.
Since platforms don’t demo well, we implemented the first AllPeers service in parallel, for digital photo sharing. Both the platform and the first app were very successful, in my view. As first releases of a complex product, they are far from perfect, of course. But they have served as the ideal validation of our ideas and vision.
At the same time, our experiences with AllPeers 1.0 led us to two important realizations. First of all, decades of combined bad experiences with multiplatform development inside our engineering team had led us to develop a Windows-only product, with the idea that we would somehow tackle portability in the future. As it transpires, this type of networked software really needs to run on all popular platforms. Otherwise, even Windows users will hesitate to adopt it if their Auntie Frieda in Herzogenaurach, with whom they want to share photos, has a Mac.
Secondly, and more fundamentally, it dawned on us that what we were really doing was building a new generation of web browser: one with a structured framework for extensibility, powerful data storage/replication features and peer-to-peer communication capabilities. Leaving aside the obvious technical challenges, it was hard for us to imagine that we could impose a new browser on the world in the teeth of entrenched competition from big, powerful players.
Luckily, at about this time (mid-2004, in my recollection) Firefox started to creep onto our radar. I realized that this might be the ideal basis for a multiplatform release of AllPeers that leveraged an existing browser with great buzz and traction. Subsequent experience with developing on Firefox has only served to strengthen this view. But I’ll talk more about that next time.
Bart from Flock was kind enough to comment on my last post, and now that we are engaged in very Web 2.0ish social software-driven discourse I decided I should play nice and not title this post “What the Flock?”, despite the almost irresistible temptation to do so.
I’m kidding, people.
Bart: I definitely agree with you that we should try to connect in person. We seem to be attacking the same problem, albeit from totally different angles, and it would be great to compare notes. I hate to be such a petulant geek, but I’m still unconvinced by the notion of packaging Flock as a browser, rather than an extension. I don’t question your sincerity with respect to not forking. But consider this: what if you build a browser with an awesome WYSIWYG editor and I build one with, say, facilities to manage and share media files? The immediate implication is that users have to choose between one or the other. I submit that this is exactly the problem that the Firefox extension mechanism is designed to solve.
There are other disadvantages to eschewing the extension route. For one thing, it’s a lot easier to convince users to install an extension, in my experience, since the perceived “cost of entry” is much lower. Your editor is great. I was really impressed by how painlessly it found my blog, downloaded my existing posts and provided me with a slick way to modify them and create new ones. I’m ready to bet a cholesterol-laden dinner in the Prague greasy spoon of your choice, however, that a) this functionality could be provided as an extension and b) someone is going to do so if you don’t. There’s a real risk of being outcompeted if this happens. I don’t see enough benefits in the soups-to-nuts browser approach to outweigh these considerations.
Anyway, my seething envy over all the advance publicity that Flock is getting has inspired me to say a bit more about what we’re actually doing here at AllPeers. Starting next week, I’ll be posting regularly about our vision and how we are going about achieving it. This is doubtless long overdue since the first Firefox-based version of AllPeers is scheduled for next month.
There’s been a lot of buzz lately about Flock. The initial, extremely vague tidbits made me a tad nervous, since I wasn’t sure how close what they are doing is to what we’re doing. Not that we shy away from competition, but since these seem like very smart guys with a great vision, I’d rather see them as friends than foes.
As more information has crept out, it’s become increasingly apparent that they aren’t competitors. It sounds like they are concentrating, at least for the time being, on a WYSIWYG editor and improved bookmarking. Both are much needed, and we aren’t doing either. Today I came across this post by Greg Yardley which points to another fundamental difference between Flock and AllPeers: whereas we are building a Firefox extension (albeit a very powerful and sophisticated one), they are actually packaging their product as an alternate browser to Firefox.
Now I’m working with second-hand information here since I haven’t seen Flock in action nor spoken to their team, but if what Greg says is true, it’s bad. The Firefox community is doing a heroic job of building, marketing and documenting an enormously ambitious project. At the same time, the challenge of driving further adoption is considerable. Those of us who share this vision need to make sure that our contributions are mutually compatible, and the best way to do this is to package them as extensions to Firefox, not fork the codebase.
I can say from hard-won personal experience that you can do amazing things with Firefox extensions if you’re willing to program to the guts of the browser in C++. It’s hard for me to believe that it wouldn’t be feasible to package Flock as an extension. And even if they are doing some fancy footwork that Firefox can’t handle, the right course of action is to propose patches that will add the necessary capabilities to the Mozilla engine.
Granted, patches take a long time (ain’t enough “o”s in “long”…) to get into the production releases, so maybe Flock as a standalone browser is an intermediate step while they wait for the necessary improvements to Firefox. (I did poke around in Bugzilla a bit but wasn’t able to find any evidence of this.) If this isn’t their plan, I hope they reconsider. Apart from anything else, I’d kill to get my hands on a decent WYSIWYG editor extension… but not if it means I have to leave the Firefox fold.
Update: I managed to acquire Flock so I’m giving it a little test run. In fact, I’m writing this in their WYSIWYG blog editor, which is definitely way cool. More to come once I’ve played around with it a bit more…
Technorati Tags: flock
So TiVo has apparently added braindead DRM to their personal video recorder. I could say so much about this, but why not quote Cory Doctorow’s Boing Boing post instead?
It used to be that it was hard to explain the TiVo. I’d tell people, “It’s like a VCR, but it’s smart enough to program itself.”
Now I’ve got a new description: “It’s like a VCR, but it it’s evil enough to screw you over if some rightsholder demands it.”
Hey, TiVo: since 1984’s Betamax decision, Americans have had the right to record TV shows even if the rightsholder doesn’t like the idea. That’s straight from the Supreme Court’s mouth. I don’t know what kind of special privilege the enteraintment industry has offered you in exchange for this spectacular display of wanton shark-jumping, but it wasn’t enough. I sold my TiVo when I left California. You can be goddamned sure I won’t be buying another one. Ever.
Note to potential TiVo competitors: MythTV is like TiVo except it includes all the features that the entertainment industry has intimidated TiVo into leaving out. And it’s free. Go make a product out of it, put it in stores, and you will sell a squillion of them. Now that TiVo’s blown its brains out, the field is wide open.
That’s probably my longest quote ever, but it’s worth it. Cory is absolutely right. It’s time for someone to recognize the business opportunity inherent in fighting the strong-arm tactics of the media industry.
I did some more reading about the eBay/Skype deal, assisted by PaidContent.org’s excellent rundown of commentary from around the web.
Rob Hof has written an extended analysis of “Why eBay is Buying Skype” at BusinessWeek Online. My takeaway was mainly that the reasons given by eBay for the acquisition sound like total bunk. I could get my visionary badge of merit taken away for saying this, but I don’t believe that voice is such an important component of eBay’s brand of e-commerce. And even if it is, why buy the cow when you can have the milk for free? After all, there was nothing stopping eBay from promoting Skype use among its users.
Jeff Clavier delves into the financial details of the deal. He quotes the cost per user at $45, which means my back-of-the-envelope estimate of $50 was not far off. (Pats self on back. Breaks arm.) My comparison was with Hotmail, decidely yesterday’s news, whereas he cites MySpace’s $20/user price as evidence that eBay overpaid on a per-user basis. Jeff’s graphic also demonstrates that their logos look kinda cool together. I suppose that might have added a billion or two to the price tag.
What really got me thinking was Skype founder Niklas Zennstrom’s comment (quoted in the BW piece) that “We can’t think of a more powerful platform to extend our vision.” And it occurred to me that the deal actually makes a lot more sense if you consider that Skype is buying eBay. I dare say the founders had other compelling motivations for selling (I picture a VC showing up at their homes brandishing a shotgun), but there are sensible business reasons as well. Whereas I theorized yesterday that it might be difficult to turn Skype users into eBay users, the converse strikes me as infinitely more realistic. As a website, eBay has a lot more screen real estate than Skype to promote stuff and a rock-solid notion of community that is lacking among the VOIP vendor’s user base.
But the main value of eBay for Skype can be summed up in two words. Or is it one word? Whatever. It’s PayPal.
So I wanted to be the first to break the momentous news that eBay has officially acquired Skype for $2.6 billion. What’s that you say? Someone else has already reported that? Everyone has?
Okay, so maybe it ain’t breaking news but I have a few things to say about this. The biggest question for me is why eBay, a company that has no obvious synergies with Skype, would be the one to take the plunge. This is particularly puzzling since the price can hardly be justified on financial grounds alone. According to the New York Times, Skype’s 2006 revenues are projected to be $200 million. Not bad, by any measure, but a thirteen-fold multiplier on forward revenues, not this year but a year hence, is pretty rich.
Are they paying for Skype’s user base? Skype has 53 million users, so that’s about $50 a pop. Not entirely out of the realm of imagination, I suppose, but consider that Microsoft paid about $20/user for Hotmail at the high of the dot.com boom. Besides which, how are they planning to turn Skype users into eBay users? Doesn’t strike me as particularly obvious.
Considering the fact that the scads of large companies that do have obvious synergies with Skype weren’t willing to pony up the cash to, ehm, outbid eBay, one might be tempted to conclude that they’ve become bored of the staid world of online auctions and paid through the nose for a bit of sexy Web 2.0 goodness. I have a better theory, presciently outlined here on Peer Pressure last week. I theorized that Skype might be in a strong position to create the universal identity infrastructure for the web. To support my theory, I drew an analogy with none other than eBay’s number one son, PayPal. And mind you, this was before I heard the eBay acquisition rumors.
Does eBay want to leverage its humungous user base, with strong reputation management features and (wait for it) built-in payment infrastructure, to create a more generalized identity mechanism? Uh oh, that almost makes sense.
Having praised Paul Graham’s last essay as both “great” and “important”, I was non-plussed to read this commentary by the usually Graham-friendly Tim Bray. Tim criticizes two aspects of Paul’s essay: 1) the problem it addresses is a straw man since no one is really advocating economic equality and 2) it is factually wrong in its claim that eliminating inequality results in a “proportional” reduction in the number of startups.
I stand by my remarks. With regard to Tim’s first point: there are hoardes of capitalism bashers out there who advocate increased equality (but who fail, in general, to explain how this can be realistically achieved). The Seattle WTO protests and associated movement were one clear example of this, and this type of attitude is particularly prevalent among the software development community. I’m reminded of a mail I posted to the XML developers mailing list back in 2002. I thought my assertion that competition enhances innovation was fairly uncontroversial, but as it transpires I was universally excoriated by the list. (I did receive a couple of private mails of support from folks who were afraid to out their capitalistic tendencies to the world.) This was a real eye-opener for me, and one of the reasons that I consider Paul’s essay to be an important one.
Tim’s criticism of Paul’s proportionality claim is unfair because it concerns a meaning of the word that I don’t believe was intended. Yes, it does have a strict mathematical usage that applies to few, if any, real-world situations. But according to the dictionary.com entry it has another, less formal meaning:
2. Properly related in size, degree, or other measurable characteristics; corresponding: Punishment ought to be proportional to the crime.
I’m sure this is what Paul meant and I’m convinced that he’s correct. The more vehement the attempt to stamp out inequality, the more marked the innovation-reducing effect.
My alterego “plasticmillion” has been hanging out quite a bit on the Mozilla developers IRC channel. The topic of the channel is set by the admins to display funny comments by participants. Today I made the cut for the first time:
That’s right, friends, I have truly arrived!
|
|
|
