AllPeers Technology Platform
A short description of the AllPeers technology platform is now online on our website. I’d be most interested in any comments (I can also answer questions here).
Bookmark to:
Old News
It occurred to me when writing my last post that I had given it a hidden expiration date, since I know already that one of the links is only going to work only for a limited time. The perspicacious among you will know which one I mean. Yep, it’s the link to Google News under the anchor text “minor hype frenzy”. The nature of a hype frenzy is such that you can’t really sum it up in a link to a single news story, so using Google New’s search results was a seeming no-brainer. The problem is that, news being news, these results will shift over time, and a visitor trawling my blog archives a month from now (implausible, I know, but I’m trying to make a point here) might click on the link and get no results at all. Some frenzy.
This set me to thinking, and I came up with a brilliant idea: a time-based search feature. This would be similar in concept to the location-based searches provided by Google Local. The latter lets you type in a location and some search terms and spits out a list of places you might find what your looking for. Handy when you’re in downtown Manhattan and have a hankering for deep-fried fish heads.
Google Time would let you do the same for a specific time period. So I could publish a link to the aforementioned “hype frenzy” query but provide extra criteria so that the results are constrained to web pages existing on September 20th 2004. Combining this with Google News’s existing capability to restrict hits to recently published stories would open up a fascinating window to the hot new stories at any given point in time.
All of this is eminently doable, of course. So doable, in fact, that someone has already done it. I’ve known about the Wayback Machine for a while as a way to look at a snapshot of a particular website at a particular time. Now it appears that they’ve added a full-text search feature called “Recall”, although when I tried it I got an unsatisfyingly blunt “connection refused” message. In any case, they’re not going to have all of those cool features for searching images, news articles, and so on. But the idea is spot on. Any chance Google could snap these guys up before my link time bomb explodes?
Bookmark to:
C is for Cookie
You remember Cookie Monster, right? You know, that blue guy from Sesame Street, the one who single-handedly cemented our nascent association between affection and fattening, sugar-laden snacks? I could write pages about the corresponding rise in childhood obesity, inevitably leading to hyperglycemia, rising health care costs, ballooning budget deficits and - eventually - the abject collapse of western civilization. But this would probably be a bit unfair to Frank Oz and the gang. After all, “Tofu Monster” just doesn’t have the same ring to it. Plus the Cookiemeister does have some rocking tunes to make up for his dietary transgressions.
Nonetheless, I do have a few choice words to say about cookies. There’s been a minor hype frenzy over the last couple of days over rumors that Google is planning to develop its own web browser. All this is based on the flimsiest of evidence, mainly the fact that they recently hired a couple of former IE developers. Hardly shocking considering the number of new programmers they are doubtless hiring with their newfound IPO wealth. But let’s play along for a moment and assume it’s the case.
In a way, this would be surprising because Google currently represents one extreme of the centralized vs. distributed conundrum, the 21st century geek-friendly version of Bud Lite’s classic 1980’s head-scratcher: “Less filling? No, tastes great!” Newish ventures such as Blogger and Gmail carry the clear message that Google feels their centralized server farm can manage your data for you better than you can by yourself. Part of this vision is the ubiquity of the web browser. Who wants to lug around a laptop when you can access your blog and email remotely? So why spoil things by developing your own browser, clearly a move towards distributing data and processing power?
Perhaps the answer is that Google realizes that despite their laudable achievements, they can’t do it all by themselves. The biggest successes in information technology have been platforms that let third parties, particularly nimble young startups, add value in a virtuous circle of innovation. The problem is that the only way to remember someone on a web platform today is through the use of browser cookies. And cookies are tied by their very nature to a specific website, due to very sensible privacy concerns. So no one except google.com is ever going to know about your Google cookies, nixing any possibility of using them for integration purposes.
All this changes with the mooted Google browser. They can still use their formidable server park for heavy data lifting, but integration can occur on the client, where they have free rein to store volumes of profile information and share it with partner services. With Microsoft totally dedicated to migrating large dollops of the web experience onto the desktop, where they can better control it, this might be nothing more than a preemptive move on Google’s part, particularly since the rumors are centered on their use of the multiplatform Mozilla browser as the basis for their own offering.
So is it true? Maybe not in the details, but it would certainly be surprising if Google didn’t come out with an increasingly robust desktop offering as time goes by. Food for thought, in any case. Make mine chocolate chip.
Bookmark to:
P2P and Piracy
It’s a bit frustrating for me to get emails from friends and family informing me of yet another bombastic RIAA offensive or imminent action by the American Congress aimed at stamping out once and for all this vile creature we know as P2P. “Doesn’t this mean you guys are toast?” they ask. In a way it’s faintly amusing to see P2P demonized in this way. It’s a network topology ferchrisakes! There are many compelling advantages of P2P architectures over more traditional networks. The fact that some of its characteristics have been exploited to circumvent the law does not mean that we should throw the baby out with the bathwater.
At the same time, maybe there is more to it than that. After all, the gun lobby uses basically the same arguments to justify over-the-counter sales of military grade automatic rifles, grenade launchers and other household necessities. “Guns don’t kill people, people kill people,” they intone. Likewise: “P2P doesn’t violate copyright laws, people violate copyright laws.” It’s certainly true that there are times we should at least consider banning a technology with some legitimate uses in order to limit the damage of its illegitimate side. So perhaps controlling P2P would actually make sense from a legal standpoint.
I’m joking of course. But even though this argument is way over the top, it does reveal an important point. In any discussion of where to draw the line between protecting civil liberties and limiting antisocial behavior, we need to weigh the pros and cons for society of the activity in question. This is why I’m disturbed by the way people tend to pigeon-hole anything and everything P2P into the same (vaguely seedy) category. If Congress are on the same page, there’s a real risk of some appalling bad legislation coming out of Washington.
The fact of the matter is that P2P is a very powerful general-purpose architecture that can be used for a lot more than pirating files. The reason that the illegal file sharing apps stole a march on everyone else is a simple case of cost/benefit ratios. Napster circa 1999 actually worked better than Kazaa or Limewire do today, thanks to its centralized index of files, and the latter are much more complex pieces of software to boot. The reason that Napster’s progeny have jumped through hoops to create fully P2P architectures is not because they couldn’t have saved themselves effort by using a simpler design. It’s because it helped them to avoid the eagle eye of the law. This tipped the balance and made P2P a viable choice for them long before it made sense to use P2P for conventional applications.
I would submit that increases in bandwidth, always-on connectivity, processing power and our understanding of network theory have now made P2P architectures worthwhile on their technical merits alone. Witness the increasing number of pure P2P applications for telephony, radio, photo sharing and other purposes. None of these activites are remotely illegal, and with a bit of luck we will see public perception of the value of P2P start to reflect this before any brain-dead laws get made. I’ll know this day has arrived when people stop assuming that the latest Justice Department raid on some socially awkward 16-year-old EMule jockey in Nebraska is going to affect our business model.
Bookmark to:
When the Eagles are Silent…
..the parrots begin to jabber. - Winston Churchill
Slashdot had a story today pointing to a couple of articles that discuss the upcoming version of MacOS. Apparently the next release of iChat includes support for the Jabber protocol. For those who weren’t paying attention, Jabber is a high-profile open source project that aims to create a universal standard for instant messaging. It has already produced several intriguing-looking draft standards.
This is a welcome trend since, at very least, it shows momentum towards adopting a common standard. At the same time, I don’t believe that this is necessarily a harbinger of things to come. Apple has a lot more to gain from pushing operability than the Windows-centric clients. Microsoft, of course, would rather keep everyone playing in its sandbox, and if Mac users feel left out, why they should just adopt Windows! AOL and Yahoo are much more interested in locking users into premium services that expanding the size of the IM market through open standards. Apple, on the other hand, wants to get people to dump Windows, and the fact that they can’t easily chat with Windows users if they do is not helping its cause.
Bookmark to:
Gmail File System
Just discovered that someone stole my idea before I even thought of it.
Bookmark to:
The Trouble with Gmail
It’s easy to be a Google fanboy. They’re one of very few companies that have managed to become stupendously successful on several fronts simultaneously. First they came up with a technical idea — PageRank — that is as simple as it is brilliantly effective. Then they created a technical implementation, in both software and hardware, that has scaled to handle billions of webpages while still serving up results at lightning speed. And finally, unlike so many of the dot.coms that came before them, they found a way to monetize their success so that revenues grow in stride with their user base. The result: a successful IPO and market capitalization of over $25 billion… not bad for a company that’s only six years old.
A lot of people seem to conclude from this that everything Google touches will turn to gold. I beg to differ. Take Gmail as a case in point. One thing’s for sure: with Google’s incredible brand and great engineering, Gmail is bound to be a hit. It’s also a good way for them to address the oft-raised issue that their core asset, searching, doesn’t have a lot of “stickiness”; i.e. it would be quite easy for people to switch to another site if something better came along. For creating lock in, Gmail will doubtless be a useful tool.
But there are problems as well. One is that it is fairly easy to game the system, bending it for purposes it was not designed for. There is already an tool that lets you use Gmail as backend storage for your blog, with email messages published on the web as blog entries. This might not be a huge win in reality, since there are plenty of free blogging services out there (including Google’s Blogger), but it does suggest other intriguing possibilities. For example, someone could launch a website called Gmail Operating System. Just enter your Gmail address and you get an interface that lets you create hierarchical folders just like a local operating system and upload as many files as you want, up to the 1 GB limit. In fact, this could even be developed to seamlessly combine multiple Gmail accounts in one interface, so you could have access to multiple gigabytes of free online storage.
One of my favorite software industry pundits, Bob Cringely, recently ruminated about the options for disaster-proof backup of his most important files. I imagine he’s not alone, suggesting that this service could become quite popular. If so, Google might end up with far more arduous hardware requirements than they initially anticipated (based on the assumption that people would use Gmail only for email), thus swelling their cost base. Plus, if their service were hijacked like this, there would be no one to click on the advertisements displayed after each email, robbing them of their only Gmail-related revenue stream.
Speaking of advertisements, I’m not convinced that Gmail is going to be as wildly successful a cash cow as the search engine itself, even disaccounting the possibility of gaming such as that described above. I’m not a big fan of advertisements, but the Google ads appeal to me for one simple reason: when I’m using their search engine, I’m looking for something. And it’s when you’re looking for something that targeted ads are most appealing. If I query Google for lodging in the City of Lights (querying “Paris Hilton”, I guess), I’d be happy to see an ad that takes me straight to a site where I can book a room. When I’m reading email, I want to read email. I’m not looking for anything, and ads are only going to distract me. This might not deter potential users, but it will certainly cut down on the ads’ conversation rate. That’s bad since they only make money when you actually click on an ad.
Bookmark to:
Bad Karma for Wikipedia?
So now Slashdot and others are pointing out the downside of Wikipedia’s open nature: it’s not as authoritative as a publication that is centrally moderated. To some degree this smacks of the typical anti-hype that gets directed at any successful technology. It’ s just too tempting to bash on the guy who’s getting all the praise and attention.
On the other hand, there is a serious point here. Several years ago, a few websites, particularly Slashdot, realized that any system that decentralizes publishing also needs to decentralize moderation if it is to scale properly. For those not familiar with Slashdot’s system: each user is assigned a reputation score called “karma”. You get more karma for posting comments that get positive ratings from moderators. Eventually, you get enough karma to become a moderator yourself, which means you get 5 moderator points from time to time and 3 days to “spend” them by bumping up or down the score of a comment. When an article gets too many comments (a frequent occurrence since Slashdot is so popular), a threshold is automatically set so that articles below a certain score (typically 3 or 4 out of 5) are not even displayed.
The system works exceptionally well, both by making really good comments more visible and by quickly relegating flamebait, spam and other crap to the virtual dustbin. The startling thing, to me at least, is that this and similar systems are not more widespread. The logical explanation is that a) these systems are hard to design and implement and b) Slashdot is a site for geeks, so by definition its users embrace a complex moderation system that might bewilder the average web surfer.
It seems obvious to me that as self-publishing on the web moves beyond the simple discussion board paradigm to encompass more complex models like wikis, decentralized moderation is going to become more and more essential. Since Wikipedia is taking all this flak for failing to separate the wheat from the chaff, and since it’s so fabulously popular, it strikes me as the perfect candidate to test a moderation system a la Slashdot, but more generalized and accessible. There’s a growing amount of academic research and informed musings on this topic.
The fragility of Wikipedia is hardly surprising given its architecture. The real (and fascinating) question is how to fix this while staying true to its philosophy.
Bookmark to:
AllPeers Digital Photo Sharing
From the shameless-self-promotion department: Our new AllPeers website is now online, and the software will be available for download soon. Watch this space.
Bookmark to:
Why Wikipedia Should Go P2P
One of the nicest things about Firefox is its support for search engine plugins. They let you search on different websites from a single edit box in the top-right corner of the browser. I have seven different plugins installed: Dictionary.com, Cambridge French < -> English dictionary, Internet Movie Database, Google, Wikipedia, Slovnik seznam (Czech < -> English) and Quote-O-Matic. I use all of them frequently. (Well okay, not Quote-O-Matic, but it seems that once you install search plugins you can’t easily remove them.)
The whole point of this feature is that you save a whole extra step when you need to search for something. Instead of having to lumber over to the IMDB website and locate the search form, you just choose IMDB in a handy list and off you go. But lately when I search Wikipedia, a webpage comes up exclaiming “Sorry! Full text search has been disabled for performance reasons” and offers to run the search through Google or Yahoo’s search engine. Kind of defeats the purpose, doesn’t it?
What’s going on here says something intriguing about free software and modern networked architectures. Why is it that Google and Yahoo are able to handle Wikipedia and the rest of the World Wide-friggin’-Web without working up a sweat, while Wikipedia can’t even handle its own traffic? Could it be the fact that it doesn’t make any money?
This is where the ideals of the information-to-the-masses set collide against the cold hard realities of capitalistism. Open source software sometimes benefits from corporate sponsorships, but primarily from the donated time and energy of legions of restless hackers. In the same way, through the work of thousands of volunteers Wikipedia has managed since its inception in January 2001 to eclipse print-based encyclopedias that have been around for hundreds of years. Unfortunately, offering your creative talents is not the same thing as giving away a full-blown server farm, so the hardware capabilities of Wikipedia have trailed the richness of its content and its increasing popularity.
It’s kind of sad that Wikipedia can get so many people to invest hours crafting articles purely to bask in the glory of seeing them shared with world, but they can’t afford to power their own site. One solution might be to abandon the “purer than the driven snow” attitude that leads them to shun any kind of commercial sponsorship or advertisements. I’m not crazy about irritating ads flitting across the screen, but if it means I can search properly then I might just put up with it.
Much truer to the Wikipedia ethos would be an architecture that lets people donate their processing power in the same way that they donate their writing skils: in small doses. The perfect way to achieve this is through a peer-to-peer architecture. Though normally thought of as a distribution mechanism for very large files, the approach is just as applicable to very popular files (including webpages). Wikipedia has more semantic information available to it than say, Google, to let it index its contents intelligently, so you get much more precise results when you search it directly. With so many devoted contributors and readers to draw on, this index could be spread around the network so that it scales to handle the increased load as more users come on board.
Bookmark to:
