AllPeers: Lessons Learned
Although AllPeers didn’t produce the kind of outcome that we had hoped for and expected, it’s been a tremendous learning experience. Hopefully others will be able to benefit from what I consider to be the main lessons.
Luck and ambition
Naturally the success of any startup is dependent to some degree on luck, and the luck factor rises in proportion to your ambitions. If your plan is to sell T-shirts online then execution is probably the main consideration. If you make really cool designs, have an easy-to-use website and do good marketing then you’ll probably make money, though you’re unlikely to be buying a private island in the South Pacific any time soon. If, on the other hand, you plan to dethrone Facebook by adding state-of-the-art social features to the fabric of the web, transforming the internet experience of billions of people, you’re going to have to execute to perfection and still get really really lucky if your company is to succeed. Of course, if you make it you’ll be assured a very comfortable early retirement.
Neither of these approaches is inherently wrong but you should be aware of what you’re getting yourself into. If you can’t stand the thought of failure, make sure you’re not tackling a problem that is too big and ambitious. In the case of AllPeers, we knew that there was going to be a lot of luck involved (as there is with any product that relies on network effects and viral adoption), and we were pretty well prepared for the challenges we would face. It is comforting to see failure in this way because we certainly wouldn’t have sacrificed our lofty ambitions to increase our chance of moderate success.
Raise as much as you can
I’m not the first one to say this, but let me express my wholehearted agreement: raise as much as you can, as soon as you can, and not a penny less. In early 2006, before we had released even a private alpha of AllPeers, we suddenly became a minor web star thanks to a couple of white-hot buzzwords (”Firefox” and “BitTorrent”) and a very positive writeup on TechCrunch. (And in fact we owe a great deal to Mike Arrington, who grasped our vision immediately and did a great job of articulating why it was exciting. It’s easy and intellectually lazy to be pessimistic before the fact and snarky afterwards, while it takes courage to go out on a limb and predict success.) We believed our own hype a bit too much, unfortunately, and didn’t take advantage of the opportunity to raise a lot of cash at a high valuation. Instead we brought in a very modest amount under the assumption that we’d be in a great position in a few short months to close a much bigger round.
As a result, we were under constant pressure to get user numbers up so we could raise more money. This isn’t the way to run a company, particularly one with an ambitious technological vision. We ended up making a string of tactical moves rather than taking a step back and looking at the big picture. As a consequence, we ran out of money before we could get the product to where it needed to be. Don’t make this mistake.
This shouldn’t be construed as a criticism of Mangrove Capital Partners, who led our series A investment round. They are a fantastic group of individuals whom I wouldn’t hesitate to recommend to any entrepreneur seeking funding, and a classic example of a VC who really does offer much more than money to a budding startup (something they all claim to do). But only a company’s founder has a single-minded focus on the company’s success, and this includes acquiring a war chest to deal with unforeseen contingencies.
Be pessimistic about the technical challenges
A direct corollary of the previous point is that you need to make a very thorough and sober assessment of the technical challenges you are facing. Make sure that you are being realistic about deployment timeframes. Then double them. In retrospect, it seems obvious and absolutely normal that it would take us the better part of two years to build a new peer-to-peer stack from the ground up and deploy it in a scalable way, especially considering that no one has built anything nearly as complex on top of Firefox before or since. But in the heady days of early 2006 we expected the product to be ready for prime time much sooner. This led to unrealistic expectations on the part of our investors (entirely our fault) and impatience on the part of our fans. It is far easier to make this type of judgment in hindsight, of course, so it’s best to be as pessimistic as possible when communicating milestones.
The viability of consumer peer-to-peer
To a large extent, AllPeers was a bet on the strategic advantage that could be gained by using a peer-to-peer network rather than a centralized server. I still feel that this was a great bet, and I don’t regret making it. As any poker player knows, sometimes even good bets don’t pay off.
Nonetheless, with all the real-world experience of building a P2P network behind me, my opinion as a technologist is that the huge challenges of deploying a consumer P2P app outweigh the advantages. The notable exception is for products that aim primarily to avoid a central point of attack (for security reasons, to exchange copyrighted works without authorization, etc.). No one would put up with the relatively crappy user experience of BitTorrent versus, say, iTunes if it weren’t for considerations of this type.
The biggest problem with consumer P2P is that other users must be online in order for files to be available. With AllPeers, we frequently heard the complaint that “someone shared something with me but when I went to download it, I got a message saying ‘no sources’.” This is intensely frustrating, especially when it is the first experience you have with a new product. Meanwhile, the cost of bandwidth and storage has been plummeting, making centralized solutions increasingly attractive.
This isn’t to say that P2P doesn’t have compelling uses. A hybrid model that uses P2P where possible and a central server otherwise looks more promising since it solves the “no sources” problem mentioned above while retaining much of the efficiency advantage of a decentralized architecture. We had already started to experiment with this at AllPeers, and this would have become a big part of our technological strategy had we had time to finish implementing it. For mass distribution of media, I believe that P2P is most effective when it is implemented at a very low-level in the network stack. Application level code shouldn’t have to worry about it, but wherever possible data should be cached at the edges of the network and delivered from the most efficient location. This is essentially how the web handles distribution of web pages, with caching at the ISP and in the user’s browser. It also underlies the technical strategy of successful companies like Akamai.
Open source/Mozilla
On a more positive note, a decision I will never regret was our choice to implement AllPeers as an open source product on top of Mozilla. I didn’t have any experience beforehand working with open source, having worked mainly with Win32 development on Windows. Nonetheless, it is no exaggeration to say that I was welcomed with open arms (pun intended) by the Mozilla community before anyone had any idea who we were or what we were working on. Recruiting new members to the cause is the lifeblood of any open source project, so newcomers are given the benefit of the doubt even if (like me) they arrive unannounced and bombard people with stupid questions for days on end before they start to get a clue.
The nature of open source software itself makes it a dream platform for any programmer. It is much easier to track down problems and understand programming interfaces when you can drill down into the source code of the platform itself. In many instances, you can gain inspiration from existing code, take it and adapt it, remix it and otherwise benefit from those who have come before you.
I am sometimes critical of what I perceive as the excessively ideological bent of many open source advocates. One of the great things about the open source movement, in my view, is that is provides a strong counterweight to proprietary software. Efficient markets have healthy competition, and the strongest innovation can currently be seen in areas where traditional software competes with open source alternatives. This is true not only of the browser market, but also of operating systems (Windows and OS X vs. Linux), databases (Oracle and Microsoft vs. MySQL) and productivity software (MS Office vs. Open Office), to name just a few. I know a lot of people who want the whole world to go open source, but I think consumers benefit most from the tension between open, closed and all the various gradations that crop up in between.
The best thing about open source is the people. I never made any friends at Microsoft grinding away at my desk with Visual Studio and Microsoft Foundation Classes, but I’ve made scores of new friends in the Mozilla community: smart, passionate, hard-working people spread across the four corners of the globe. Working with open source is a rare opportunity to gain a competitive edge in the technology business and have fun doing it. I’d recommend it unhesitatingly to any software entrepreneur.
5 Comments »
Trackback URL RSS feed for comments on this post.
Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>
Sorry to hear about AllPeers closing up shop.
Your experience could probably help another open source project, Metalink, which deals with downloads. It’s a bit different, but I think you’d be interested in it.
Comment by Ant Bryan — 3/5/2008 @ 7:31 pm
Hey Matt,
very interesting post — it is rare to get this kind of inside view!
So, thought I’d offer my thoughts on some of your points, both technical and not. Its been some time since I’ve worked as a network engineer but that was during the heyday of p2p, so the technical points might still be relatively well-informed. The rest is pure opinion
Regarding the technical challenges, I’ve often wondered whether one reason you tackled so much at once was that you intended to maintain a degree of control over the network. Much of what you did, protocol-wise, appears to be incremental improvements over XMPP and BitTorrent and there seemed to be no really obvious technical reason to require use of your own server and not to federate.
This has obvious implications for the growth speed. I suspect that, even though you maintain that Jabber is for nerds only (but see GoogleTalk), it would have increased the speed of growth tremendously if people could have re-used their existing accounts. Particularly given the sorry state of file transfer in most other XMPP clients
That said, a closed network is obviously one of the things that makes it hard to achieve network effects and viral adoption. Open sourcing didn’t change that because AllPeers was still a different network. I would suggest that while luck certainly is a factor in achieving these kinds of effects, it is not attributable only to lack of it that you didn’t.
To make this perfectly clear: I suspect that building your own network was not necessary and that the attempt to do was one, if not the, main factor in AllPeers demise. btw, GoogleTalk is also a good example of how one can build onto XMPP but still maintain brand identity.
As a corrollary, if you wouldn’t have had to invest the effort in doing that, you might have had some time to counter the problems with consumer p2p (which I completely agree with, as you know from my server post
To also end on a more positive note, I’m really happy to hear about your positive experiences with the Mozilla community and open source at large. What a rebound from that Schemantix situation!
Cheers, Ingo
Comment by Ingo — 3/8/2008 @ 3:18 pm
Thanks for your comments, Ingo. You’re correct that one of our motivations for setting up our network was to maintain control and therefore, indirectly, create value. I’m not sure how much value Google has really created with Google Talk, compared to, say, Skype.
From a technical perspective, it was always my understanding that XMPP is a client/server protocol, although they were/are working on P2P extensions. Am I wrong? Both our application-level wire protocols and authentication mechanisms were similar/identical to XMPP, but I don’t see how using XMPP would have obviated all the work we put into our P2P implementation.
To summarize: the biggest reason we decided to set up our own network was that we weren’t able to find anything else that fit the bill. Definitely not anything open source.
Comment by Matt — 3/10/2008 @ 9:11 pm
On the question of value: I’m not sure how one can value an instant messenging platform but I would suspect that GMail does create a lot of value and I would also surmise that IM and mail are going to converge, so it was a logical step.
btw, last I checked, Skype wasn’t doing so good either — if you look at the revenue. Looking at the dough the founders made is something else, of course
Regarding the technical advantages: I wasn’t trying to suggest that you shouldn’t be using BitTorrent for fast P2P downloading but was rather trying to say that, firstly, there could have been savings in implementation time through re-use of existing technology (i.e., XMPP servers and client libraries). Obviously I don’t know how much of your development time was sunk into that part but I guess it would have provided a nice jump-start, to build upon. Secondly, there would have been a large user-base right right at the outset (everyone on federating XMPP networks) with XMPP compatibility and also more potential for making people aware of your client.
For this latter argument, envision the following: You’re already using AllPeers and you send me a file to my regular XMPP client (Pidgin, in my case), using the regular XMPP file-transfer (which is a mess, but thats another topic and certainly something a dedicated group could have made a difference on). You would accompany the transfer offer by a message saying that I could get — through using AllPeers — faster downloads, thumbnail preview, and so on. In many cases, given the sorry state of file-transfer in existing XMPP clients, that alone would have made the switch interesting. Now, the switch itself would also have been made easier by the fact that AllPeers could have just re-used my already existing account on the other Jabber server, instead of requiring a new sign-up.
Last, but not least, you might have gotten people who want IM in Firefox and I also guess it wouldn’t have been hard to improve upon the GUIs of most current IM-client using Firefox UI technology.
Granted, these are all small changes, many potentials and so on. However, I reckon AllPeers wasn’t that far from achieving networking effects and these small changes might have been all that it took. That, in the end, is what I was trying to say with all this. Whether it was with XMPP or something else, by creating your own network, you missed out on networking effects that a larget network would have given you.
Hope this was somewhat intelligible. I’ve been reading too many papers today
Comment by Ingo — 3/11/2008 @ 9:23 pm
Yes, that makes sense.
If I were going to do it all again (which I haven’t entirely ruled out), I would definitely want to leverage some existing network far more than we did. Using the existing infrastructure for XMPP presence and messaging make sense. This would have enabled us to concentrate on the P2P file transfer bit in the same way SIP signaling can be layered onto XMPP. Interoperability with existing XMPP chat clients would be an added bonus.
I also believe that we would need much better integration with email (for some reason our current approach seems to baffle most users) and the web. Someone dubbed this “P2Web”, pointing out the growth would be far strong if a custom client were only needed for sharing, not receiving files. Of course, you would probably have to pay for large file transfers in that case, so the motivation for both parties to get the P2P-enabled client would be there.
Comment by Matt — 3/13/2008 @ 11:22 am