I had an interesting conundrum to solve today. Whereas in AllPeers 1.0 you have to explicitly load and store resources in the database, in the new version persistence is implicit. This means that you can ignore the existence of the database and assume that any time you change a resource it will be updated on disk automatically. The difficulty is that you might have several databases associated with different user profiles, so Firefox needs a way to determine which one to use.
The obvious solution is to use Firefox’s built-in profile manager, which associates a separate directory with each user. But in a P2P environment the client application can also be a hub (aka “supernode”) responsible for managing shared data of various types. The hub has to have its own database, so if my strategy is to work, it also needs to have its own user profile. Not only that, it needs to reside in a separate process from normal Firefox since both will be running simultaneously.
I started to investigate whether I could have AllPeers launch a second instance of Firefox to run as the hub and immediately ran into a problem: the hub shouldn’t display another browser window, and I couldn’t find any obvious way to run Firefox without a visible window. So off I headed to the IRC channel to beg for help. One of the Mozilla core developers, Benjamin Smedberg, among others, encouraged me to use XulRunner instead of Firefox for our hub application. But this would mean a lot of extra baggage for AllPeers, so I set out to find a solution to the windowless Firefox problem, encouraged by Gertner’s Third Law: Anything’s possible when you have the source code.
The whole incident got me thinking about XulRunner and Firefox in general. I’ve been leaning towards the idea that Firefox, rather than XulRunner, should be the core runtime environment. Thunderbird and other XUL applications (such as my putative IDE) would run inside Firefox, which would become an alternative to the tired desktop metaphor for GUI apps.
On the other hand, what makes the Mozilla folks so exceptional is how effectively they’ve navigated the strategic and marketing challenges — as well as the technical nitty gritty — of creating a new platform. Just look at how they maintained focus by putting the Mozilla Suite to pasture over the yelping of a million rabid Slashdotters to “stick a fork in it.” So I figured I was probably missing something and spent some time investigating the issue, stumbling upon a blog post by none other than same said Benjamin Smedberg.
I think I get it now, and maybe I was wrong. The vision looks to be precisely what I was hoping to get from future versions of Firefox: XUL apps, if they so choose, will be able to run in the same process (and potentially in tabs of the same window) using the same runtime. The difference is that XulRunner strips out some of Firefox’s not-entirely-essential baggage, like bookmarks, so the core engine is kept as lean-and-mean as possible. Hard to argue with that.
Oh, and I did get the windowless Firefox thing to work.
Marc Boucher has generously taken a crack at cleaning up and improving the quick-and-dirty hack I call “Slashdot Live Comment Tree”. I’ve made his emails, which describe the process in detail, available on this blog. In his last mail he says:
I’ve added a bug in the previous version. I had noticed that the script hides the <td> containing the message info, not its parent <tr>. But after adding the .parentNode I forgot to also add it at 2 other places (”Collapse All” and “Collapse Main” when checking for style[”display”]).
In attachment 2 versions of the script. One that hides <td>, the other hides <tr>. This is only a cosmetic choice, the hidden titles are 2 pixels closer when <tr> is hidden. Give me your opinion.
My aesthetic sense is pretty much non-existent, so I’m posting both versions here in case someone more qualified has an opinion:
Script 1
Script 2
I’m still using the old, pre-security crisis, version of Greasemonkey, so don’t know how these will function in the latest version. Caveat surfor, as always. I’ll give this a little time to percolate and then put the winning script up in place of the current one.
In other Slashdot/Greasemonkey news, Julien Couvreur has created a cool script called “Slashdot - Single Page View” that injects subthreads of Slashdot discussions onto the main comment page, instead of making you open a new page. It looks great when used in conjunction with LCT!
Okay, maybe advertising isn’t dead yet, but I did see a couple of articles in the past couple of days that suggest there is a producer-driven trend towards less advertising, at least of the chain-you-to-your-couch, pry-your-eyes-open Clockwork-Orange-style variety.
One article is from the Financial Times, so you can’t read it unless you’re a subscriber. (This is stupid, by the way, because I wouldn’t have hesitated to pay a small pay-per-download charge after it was recommended to me by friends. Instead… well, let’s just say that I picked up the print edition.) Just in case you are, here is the link, with an excerpt for those who aren’t:
“Advertisers are starting to find television a turn-off”
Meanwhile, the search for alternatives to television is picking up. ZenithOpti-media, an advertising buyer that forms part of Publicis, said this month that television’s share of the global ad market had probably peaked last year. As recently as April, it had predicted television would gain market share this year and next, before heading lower.
The other is from Fortune, which is similarly restricted to subscribers. (The fact that these publications try to force you to subscribe to the print edition in order to access online content, instead of just charging a per-article fee, is a perfect example of how their bricks-and-mortar mindset is preventing them from taking full advantage of the internet… and opening the door for someone more flexible to eat their lunch.) At least this one has a longish intro that is available for free: “Innovation: the Future of Advertising”. The article looks intriguing, so I’ll have to find a way to get my hands on it. I can’t even pick it up at the newsstand — not yet, at least — since I’m in Europe.
Another article in Monday’s FT ties in nicely with my rant about cinema advertising:
“Flash mobbing gives up its wild past”
It turns out that the [supposedly spontaneous “flash mobbing”] concerts are being staged by Ford Motor with Sony Pictures Digital to promote the launch of the new Ford Fusion car, which Ford wants to portray as cool.
Trust big business to turn flash mobbing to its advantage.. But what puzzles me is where are the howls of outrage? You would have thought that people in their 20s and 30s would have been sickened by the sight of a company co-opting their little gesture of anarchy into the service of a branding strategy. But the artists themselves — The Roots, Fat Joe, The Wallflowers — are clearly happy enough to perform for Our Ford and the fans seem thrilled at the idea of attending a musical sales pitch if it means they get free tickets.
I’m starting to think that its time to start an activist movement aiming to increase awareness of the hidden costs (financial, aesthetic, moral, etc.) of marketing run amok. The second FT article goes on to mention Naomi Klein’s anti-brand screed, No Logo. But Miss Klein is a knee-jerk anti-capitalist in the brick-throwing Seattle-WTO-protest mold. So while I sympathesize with some of her views, I can’t take her proposed alternatives very seriously; where they’re not hopelessly vague they’re hopeless utopian. What we need is an honest-to-goodness red-meat-eating capitalist take on the need for restraint in marketing, for the sake of both producers and consumers.
A long time ago, in a New England town far, far away, I first started using computers to write documents and code. One of the first lessons I learned was to save my work early and often. For someone as hyperactive as me, there’s nothing worse than bashing out pages of text only to lose it all due to a system crash, mistyped command, pet yanking out the power cord, soda spilled on the keyboard, Act of God, alien invasion or any of the many other misadventures that conspire to reduce our productivity. I hate - absolutely hate! - having to redo work after believing that it was all squared away.
Over the years I’ve become a totally compulsive saver, my fingers daintily whacking Ctrl-S after every sentence or (not uncommonly) every word. Writing in the web browser (this blog, for example) is thus an exercise in frustration, since the saving process is so intricate and time-consuming. Save as draft. Reload. Find your place in the text box. Wash. Rinse. Repeat.
So imagine my excitement when I discovered the Scribe extension for Firefox. It adds options to the File menu for saving and loading form fields from disk, and even overrides Ctrl-S so peace of mind is always a keypress away. Unfortunately, when I tried to use it on a blog entry, it happily saved my text but refused to load it, complaining about a “parser error”. Today I finally got around to poking around in their code, and I discovered that the problem is that the file is stored with a faulty character encoding. Since WordPress uses a bunch of weird non-ASCII formatting characters, this broke the XML parser that tries to read the text back into the web form.
To make a long story short, I fixed the problem and have been happily and compulsively saving web forms all day. So maybe geeks don’t get laid in high school, but being able to hack other people’s stuff almost makes up for it. I reported the problem and fix to the Scribe folks, so hopefully they’ll work it into the next release. In the meantime, if you’ve encountered this problem, drop me a line. I’ll send you the corrected version.
My theory that obnoxious movie advertising is on its way out was challenged twice in the past few days. First of all, I noticed this tidbit in the IMDb Studio Briefing, the source of some previous evidence I cited hinting at a coming revolt against longer and more intrusive ads at the cinema.
I’m not convinced that a lack of active complaints against these ads is proof that they don’t get on people’s nerves. On the other hand, I went to see Sin City yesterday, arriving early to have a drink with my friends in the cinema cafe. I was contentedly sipping my second whiskey (yep, here in Prague we get booze in the movie theatre) when my companions started fidgeting and complaining that we were going to be late. “Don’t worry,” I said. “They show tons of ads before the movie starts.”
“But we like the ads,” they replied. Bah!
I still suspect that this reaction has more to do with the trailers they show before the movie than the spots for McDonald’s and Coca Cola. I’m interested in other opinions, though. Is it me? Am I just a whiny curmudgeon? Or are other people annoyed at being forced to sit through irrelevant product advertising after forking up big bucks for a night out at the movies?
I’m still fully engrossed in our Firefox development. I managed to turn our stuff into an installable extension today, so I should be able to start pushing pieces out onto this blog as soon as we work out the licensing issues.
I’ve said plenty about the potential of Firefox to transform the face of computing. One of prerequisites for this is to coddle software developers in the same way that Microsoft has learned to. Though I suppose the Linux crowd would sneer at this, I find MS Visual Studio to be jaw-droppingly impressive: a highly functional editor, great debugger, extensible architecture, etc. But then I’ve already laid my cards on the table with regard to turning Firefox into an IDE.
The other asset Microsoft provides to programmers is the Microsoft Developer Network (MSDN). This is an enormous library of knowledge base articles and code samples. I remember, in the early 90’s, how I used to waste hours on the phone with MS technical support waiting for some call center phone jockey to help me with problems that I generally knew far more about than he did (they were always men). Well that hasn’t been necessary for years thanks to the internet and MSDN.
Firefox development resources are much less expansive and well-organized, spread as they are across a host of websites run by the Mozilla Foundation and various third parties (XULPlanet, MozillaZine, Mozdev.org, etc.). The good news is that the community is well aware of this issue and is making a concerted effort to turn their developer site, Devmo, into the equivalent of MSDN. In fact, they just changed the site’s URL from developer-test.mozilla.org to developer.mozilla.org, evidence that the site’s maintainers feel that it is coming of age.
Another great resource that I found out about yesterday is the Firefox developers’ IRC channel. This was a discovery of “how did I ever live without it?” proportions. All the Mozilla core developers seem to hang out there, and they’re amazingly willing to lend a hand when a newbie like myself pops up and starts asking idiotic questions. I’ve also been urged by just about everyone I’ve talk to there to make contributions to Devmo. With the power of the nebulous Firefox community now focused on improving developer documentation, as well as the product itself, I’m confident that newcomers to the Mozilla development community will find it increasingly painless to get up to speed.
I mentioned in my last post that I had written a parser for RELAX NG accessible via Mozilla’s XPCOM. I imagine that this caused many an eye to glaze over. What, after all, is RELAX NG, and why should we care? Glad you asked.
The generally accepted way to describe XML vocabularies is the World Wide Web Consortium’s XML Schema. XML was experiencing the white-hot glow of hypedom when this standard was created, with dozens of companies striving to get their finger in the pie. The result was predictably a classic example of design-by-committee: big, bloated and ugly (but, other than that, a fine effort). A few brave souls saw this coming early on and tried to head the W3C off at the pass by producing competing specifications that lacked these deficiencies.
The most successful of these efforts was RELAX NG, a hybrid of two earlier schema languages, Murata Makoto’s RELAX and James Clark’s TREX. (James, in particular, is a legend in the markup world. At the first SGML conference I attended, in 1996, a panel discussion was delayed by several minutes when his name was read off, leading to a spontaneous standing ovation before he even said anything.) RELAX NG is a perfect demonstration of a standard produced by a small number of brilliant individuals: concise, elegant and powerful. The opposite, in other words, of XML Schema.
With the full weight of the W3C behind it, XML Schema has achieved much broader adoption than RELAX NG. Standards, after all, aren’t much use if no one’s standarizing on them. For the same reason, we opted for XML Schema in AllPeers 1.0. After many months of putting up with its weaknesses, I decided that we should put an end to our misery and create our own ad hoc resource description language for AllPeers 2.0. Our chief architect, Honza, urged me to look for a solution that was at least semi-standard before going proprietary. I immediately went back to the almost-forgotten RELAX NG spec, and I really liked what I saw. I whipped up a parser and haven’t looked back since.
The morale of the story is that successful standards aren’t cooked up by a committee of chefs. The same phenomenon can be observed in the web services world, where the grotesque potpourri (French for “rotten pot”) of SOAP and WS-Everything has finally given rise to an aggressive backlash by those clever enough to know that there is a better way. Speaking of which, Tim Bray announced today that Atom 1.0 is about to roll off the presses, with a schema described using (wait for it!) RELAX NG. This makes so much damn sense that I’m going to go out on a limb and predict a major resurgence for RELAX NG. You heard it here first.
As Garfield (the cat, not the president) used to say, “diet” is just “die” with a “t” on the end. Similarly, “deadline” is just… well, you get the point. Bottom line is that we’ve been slogging away on the new version of AllPeers, and after a heroic effort we completed our first integration milestone yesterday. (To protect innocent, I’m not going to say when said milestone was originally due.) Hence my nearly week-long hiatus from blogging.
The existing version of AllPeers is based on Microsoft’s development libraries and runs only on Windows. Regular readers of this blog may have guessed by now that the new AllPeers version is based on Firefox. Obviously one of the key advantages of this switch is that we will run on pratically every platform of any significance. A more subtle benefit is that we integrate more smoothly into the web browser environment. AllPeers 1.0 is a web browser embedded in a peer-to-peer client. AllPeers 2.0 will be a peer-to-peer client embedded in a web browser, and this turns out to be a far more compelling proposition.
In recent years I’ve tended to alternate every few months between two entrepreneurial pursuits: programming and begging for moneyfundraising. Software development is intensely intellectual, often frustrating but commensurately satisfying when you finally get the beast to run. Raising money involves travelling to exotic places, meeting interesting people… and trying to stiff them with the restaurant bill. So both have their appeal, but I can say without hesitation that I’ve had more fun over the past few weeks, hacking Firefox, than I have had at work for years.
I’ve therefore decided to share this enjoyment with the rest of the world. I’ll be writing regularly about the AllPeers 2.0 development process, both on a philosophical and technical level. I’ll also be augmenting Peer Pressure’s download section with some of the code that we’ve been creating. Currently in the pipeline, among other things, are XPCOM components for an XML pull parser, an XML pretty printer and a RELAX NG parser. Modern software development is all about releasing early and often, and we’re keen to jump on the bandwagon. Hopefully this code will be useful to someone, and ideally we’ll improve the quality of our product by finding out quickly what works, and what still needs work.
Had a great Web 2.0 experience today (how I wish there was a more elegant term for this!). I was reading my feeds in Bloglines, one of which is the del.icio.us/popular feed (which I highly recommend). Among the top links was a Wikipedia article about, of all things, yesterday’s horrific London bombings.
The first thing that struck me was how amazing it is for an encyclopedia to have a long and detailed article about something that happened so recently but which is clearly destined to be of historic importance. The second thing that struck me was how I had found this article: not by searching but by tapping into the collective consciousness of the del.icio.us community. And the third thing that struck me was that I had accessed this information not on the del.icio.us website but in a third-party blog reader. Basically every aspect of this experience would have been hard to imagine a mere five years ago.
A telling encapsulation of the future of the web if ever there was one.
Okay, he did no such thing. But this blog post does seem to indicate that I’m on to something.
I went to bed last night with a glow of satisfaction, having been received numerous plaudits for TechnoProxy, several generous contributions to the graphic design (which evolved rapidly from ridiculous to sublime) and even a link from Boing Boing. I woke up at 8:30am CEST to several angry comments, posted during the night, from people who couldn’t get the script to work. I tried it myself and, sure enough, Stef’s web service was returning empty documents for sites that I knew had Technorati links.
I figured that his API key had run out of juice, and sure enough, after showering and getting dressed I tried again (at 9:10am or so) and the script was working fine. I assume that Technorati runs on California time and resets the keys at midnight — 9am here in Prague.
This mirrored a recent discussion on the Greasemonkey list about the new Google Maps API and their wacky access system, which requires that you request an API key for a specific domain. It’s so specific that a key for www.foo.com apparently won’t work for plain old foo.com. This has caused no end of grief for normal server-side web programmers and it’s obviously uncool for client-side scripters who might be riffing off any given domain at any given time.
These two incidents confirm what I already believed: the API key system is broken and has become one of the primary hindrances to a more structured XML-based web. Back when the system was invented (by Amazon?) there was no Greasemonkey and very little client-side coding in general. So it probably seemed like a good idea to throttle the user’s requests by identifying the programmer, since in a web application environment they tended to be one and the same. With Greasemonkey, the programmer is very likely not to be the user. What’s more, if you do want to use an API key in a script you have to release it to the world, which explains why TechnoProxy has to go through Stef’s web service instead of communicating directly with the Technorati server.
The solution is to handle XML requests in the same way as HTML requests. With AJAX it’s just as easy to bring a site to its knees by pounding it with HTML requests, so it’s hard to see the justification for treating the two differently. If someone is overwhelming a server with XML requests, it should return 503 (Service Not Available) and lock the client out for some time period. This is the approach taken by del.icio.us, for example. So here’s hoping that the API key suffers a rapid demise. It can’t come soon enough for my taste if client-side coding is to reach its huge potential.
Stef Magdalinski and I have been at it again. This time we’ve put together a Greasemonkey script that inserts information from Technorati onto every webpage you visit:
Download TechnoProxy here
The script works in much the same way as WikiProxy: Greasemonkey Edition. When you surf to a page, my script sends an asynchronous request to Stef’s web service. He then queries Technorati to see whether anyone is linking to the page, and if so sends back an XML response. That’s right kids: AJAX! When the response is received a little green Technorati icon is displayed in the upper lefthand corner of the screen. Click on the icon, and a frame opens in the page, displaying the links. You can close the frame with the ugly red X in the top right.
Speaking of ugly, the biggest problem with the script is that Stef and I both turn out to be utterly devoid of graphic design skills. Want to see your name in lights? Would you settle for a mention on a blog or two? Fix the design so it looks better and more consistent across different webpages, and you’re in!
Update: TechnoProxy has been fooseled! I’ve put up a much prettier version of the script, as modified by Gina Häussge. Thanks, Gina!
Update: Thanks to Prakash Kailasa for suggesting a fix for the Z-order problem:
Add a “z-Index: 999;” to the style rules for the iframe. Prevents it from being hidden by other elements (visit
http://blakeross.com/index.php?p=152 and click on the Q icon to see the problem).
Update: Jeff Gates created a PNG version of the Technorati icon that renders great on dark backgrounds. Thanks, Jeff! Is this what Stef meant by the “LazyWeb”? Pass me a marguerita, I could get used to this…
Update: If you can’t get the script to work, this is probably because Stef’s API key has expired for the day (see my latest post). We’ll see if we can find a solution for this. Stay tuned.
A hilarious anecdote from Firefox wunderkind Blake Ross. It’s heartwarming to hear to what extent the older generation has engaged in the latest browser wars, at least when their descendants are leading developers on the project. Laughs, heart, intergenerational family ties, Middle Eastern religious leaders, police escorts… hmmm, shouldn’t Hollywood be all over this story?
It never occurred to me before that the word “smut” has Germanic etymological roots. To me it has and will always evoke Tom Lehrer.
Perennial favorite Bob Cringely also weighs in on the Grokster decision. In this case, however, Bob seems to be motivated mainly by a desire to rise above the inane blather of the entrenched interests on both sides of the debate by finding something — anything — counterintuitive to throw into the ring. He claims that, with pesky Grokster and co. out of the way, the media companies are now free to accelerate the roll out of their own online media operations… and he predicts that they will do so between now and the end of the year.
I believe that the opposite is true. The problem with big media is that its most influential folks are tied to old-style models of media distribution. Who are you going to listen to, your veteran EVP of Worldwide Sales, responsible for your $5 billion annual turnover, or your Director of Online Media, probably a wet-behind-the-ears 20-something fresh out of college and sporting a ponytail, low-slung blue jeans and a $300 pair of sneakers? No, I think that, if anything, the decision is going to lull the media folks into complacency by providing them with some (illusionary) breathing room. With the pressure of piracy now reduced, why rush to cannabilize your revenues in the pesky free-for-all of the internet?
Where Bob hits the nail on the head is in his implicit assertion that P2P is about more than efficient downloading of media files. Moving processing power to the edge of the network makes it possible to aggregate and process structured data on the client, so value can be added to existing data streams in a decentralized manner. He doesn’t believe that the “little guy” will be involved in creating these structured feeds, however, claiming that metadata and markup are too “laborious” for Joe Average. Instead, he sees the future of the web in the repurposing of XML APIs published by Yahoo, Google and the like.
I disagree with this as well. RSS is, after all, just an XML API that is standardized enough to be generated by turn-key blogging software. There’s no reason why this principle couldn’t be extended to data other than blog posts; this is hinted at by efforts like Structured Blogging. Anyone can fill out a form describing a calendar event, media file metadata or whatever. At that point the data’s already structured, and it’s trivial to generate an XML feed from it using standard tools. Nor do I believe that the necessary incentives are lacking. Just make sure that the data is useful first-and-foremost to the users themselves, and the added benefit that comes from aggregation is just icing on the cake. Modern tag-based architectures like del.icio.us and Flickr are perfect illustrations of this principle.
So I see the web evolving in the direction that Bob postulates: slightly fatter clients (he even mentions Greasemonkey) squeezing value out of XML data feeds in weird and wonderful ways. But the most revolutionary impact of the internet has been to put smart, creative individuals on something close to even footing with the media behemoths. As we shift from HTML to XML, this phenomenon will be amplified, not diminished.
Commentary about the Grokster case is everywhere. One of the most notable for me is in this week’s Economist. Like much of the more level-headed analysis, the article (buy it online or on a dead tree near you) finds that the Supreme Court “did a reasonable job” in striking a balance between protecting copyright and encouraging innovation. The striking thing about their take, however, is what follows. The Economist has a reputation as a conservative rag, but it is actually liberal or even libertarian in much of its philosophy. The article rips into seemingly endless extension of copyright, originally designed to ensure that artists were compensated for their work, but now tailored to boost the profits of large media corporations long after the creative types are death and buried:
A first, useful step would be a drastic reduction of copyright back to its original terms –14 years, renewable once. This should provide media firms plenty of chance to earn profits, and consumers plenty of opportunity to rip, mix and burn their back catalogues without breaking the law. The Supreme Court has somewhat relunctantly clipped the wings of copyright pirates; it is time for Congress to do the same to the copyright incumbents.
I just dubbed this Gertner’s Second Law: any time the Economist agrees with Larry Lessig, they must be right.
