Just because we use certain tools very frequently, doesn’t necessarily mean we understand them as well as we could. Often, we use patterns we know will work, without really understanding why they work. This is certainly the way I use some of the more obscure parts of Javascript.

And so, after finding myself with some free time, I decided to sit down and think more deeply about a topic I generally try not to think too hard about. The topic I chose was the relationship (which previously I merely suspected to exist) between scope, garbage collection and closures in Javascript.

The results of my ruminations are collated in the presentation embedded here. My understanding of this area is now much clearer than it was before. I published the presentation in the hope that other people might find it useful too.

Needless to say, it’s easier to read in full-screen mode.

Zooming In
This is the second in a series of articles on the LM micro-component architecture. Click here to start from the beginning.

In this post we’re going to describe components in more detail, and use a real application to show the micro-component architecture in action. For the sake of clarity, we’ve simplified some of the details, but the underlying model as described here is essentially identical to that of the functioning framework.

Within LM, components are either atomic in nature, or else they are composed of other components. Atomic components can either be concrete (HTML, CSS, JS or content) or else they are abstract. More on abstract components later.

The image below shows a typical component breakdown for an interface button. It can be seen that the button is a composite component, and, even though it represents a very simple interface item, it is itself composed of quite a high number of sub-components. At first glance, it may seem like overkill to break something as simple as a button down into so many parts….

… but if we consider (image below) a component representing a button group, we can immediately see the low-level factoring that occurs naturally. It should be clear that both graphical and behavioral consistency are reinforced by explicitly specifying the component breakdown in such detail.

A Real-World Example
We can see that the images below describe two folder managers which share the same general structure. At first glance, they appear more alike than they actually are. Each folder manager deals with a different underlying data type (images or contacts) and behaves differently depending on the context (double clicking an image fires-up the image viewer, whereas double clicking a contact brings up the contact properties form). That said, they also have a great deal in common. Apart from the graphical layout, many behaviors are also shared: dragging and dropping has the same effect in both contexts, as do the cut, copy and paste buttons, for example.

The image below shows a breakdown of some graphical components in the generic folder manager. The folder manager also references many other types of components, but it’s harder to visualize them graphically. The highlighted parts represent the various button groups (as described above) and the various icon templates. We have already seen that button groups share a wide range of common components. Similarly, behind the scenes, both folder managers share many items. For example, the same drag and drop library is used by both instances. Combined with the fact that they both use the same type of data structure to represent two quite different hierarchies, this means both instances are highly compatible with each other. Although it doesn’t make much sense to drop an image into a contact folder, if both folders contained similar data types, it would be straightforward to drag and drop items between the two windows (with both source and destination data structures being automatically updated both in the bowser and on the server).

E Pur Si Muove
And just to show that these examples do indeed come from a working application, below is a two minute video which shows the contact and image managers being used within a Twitter client which is written in LM.

It’s all about Structure
By now, it should be clear that intense factoring is one of the defining principles behind the micro-component architecture. And, although the LM compiler does carry-out some additional factoring, the key point is that the architecture itself promotes the natural identification of recurring structure at the specification level.

The first post emphasized the high level advantages of this granular component sharing: increased inter- and intra- application compatibility, leading to the possibility of complex application ecosystems emerging, in which applications can be combined, re-combined and augmented with ease.

In the next post, we’ll look at some low-level advantages of the micro-component architecture. We’ll see how it allows highly complex applications (comparable to desktop software) to be delivered to the browser in a highly efficient manner, even over high latency (eg mobile) networks.

The Big Picture
LM Framework uses a software architecture in which web applications and services are designed and delivered as a series of generic building blocks, which we call micro-components. This is the first in a series of posts which together will describe the architecture, explain the rationale behind many of the design decisions, and point-out why we’re excited about its potential. The series will start off at a high level, and will become increasingly more detailed and implementation orientated. In this post we’ll introduce the Mixable Web - we’ll explain what it is, and how it could possibly influence the way we build and use internet software and services in the future.

The Mixable Web is simply an alternative way of viewing software and data, and is very general. As such, it applies both to how users interact with applications, and how applications themselves are constructed. It hinges on the notion of compatibility. The logic behind the Mixable Web is very straightforward: the more applications (or more precisely the components from which they are built) are compatible with each other, the richer the user experience, and the more power in the hands of the developer.

Size Really Does Matter
The image below shows a stylized component breakdown for an application. It’s not necessary at this stage to know what the application does, or what the components represent. The key point is that a high proportion of components are referenced more than once within the specification. This is what facilitates the underlying compatibility between the various parts of the application, and, as we’ll see in a moment, it will also enable us to create new applications by simply combining existing ones.

The skeptical reader is probably thinking “So what? There’s nothing new here”. And whilst it’s true that component re-use is an established practice in software engineering, it’s also rare to see the concept taken to such extremes (in LM, components can be as small as a single image, or a line of text). In future posts we’ll list some low-level advantages of the micro-component model (which are closely linked to the current limitations of both the browser, and of delivering a web application as a series of distinct web pages). But for the time being, we want to focus on the high-level issues concerning deep component compatibility - and the resulting benefits to both the user and the developer.

In the image below we can see what happens when two applications are combined. It can be seen the two stand-alone applications share many components. This is not a coincidence. The smaller and more general purpose the building blocks, the more likely they can be re-used in different contexts, which leads to a more granular compatibility both within and between applications. This also explains why the original multi-purpose Lego blocks are much more versatile than today’s highly-specific ones!

We’ll be seeing this image again in a later post where we talk about compression and latency - two key issues for web-hosted software (SaaS). There are many other reasons for being happy to see so many shared components in the new combined application. Setting aside the footprint issues mentioned above, here are some of them:

• Graphical consistency
  The combined application shouldn’t look like a patchwork quilt
• Behavioral consistency
  Clicking on the same button in different contexts has the same effect
• Customization consistency
  The developer (and in theory the user) can customize components.
• Data consistency
  It should be possible to share data between the two sub-applications
• Easier development
  Using existing components represents less work than creating new ones
• Less testing
  Existing components are more likely to be stable

The fourth point, data consistency, has major ramifications. One of the key differences between desktop and web applications is their relative levels of interconnectivity. Desktop applications are more like islands: information can travel between them, but the journey is often not easy. Web applications, on the other hand, frequently have data integration at their core - this a direct consequence of the web’s open and multi-point architecture. Much has been made of ‘mashups’, where data from diverse sources is presented on the same web page, but this still represents a mainly read-only situation. In the future, it will be possible not only to read, but also share, organize and integrate information from disparate sources, all from within the same application.

Application Ecosystems
Just about every web site connects to a some sort of database. Most of the time, the only way to access the database is via the website, although some services now make a distinction between the two (Twitter is a good example). The micro-component architecture allows data sources to be viewed as components themselves (each with its own access permissions). And since LM applications share so much low-level DNA, it will be possible to seamlessly integrate information from different sources within the same application. This means going beyond simply displaying the information on the screen, and transforms applications into something closer to mini-platforms - with the difference that they do not necessarily have to remain isolated. To return to a geographical analogy: an application can start off as a small island, which is then gradually integrated with other applications to form ever larger and more complex land masses: bigger islands, continents, and - why not? - even planets!

This potential for combining and re-combining functionality and data sources into ever richer (or more focused) applications is exciting. There are some pretty clear evolutionary analogies, which open-up the possibility of dense application ecosystems emerging. But let’s not get ahead of ourselves. To flesh-out these concepts, we’ll present two examples; one generic and one specific.

In the image below we can see three stand alone applications. For simplicity we have each application referencing only one data source, but there is no reason why they should not reference more. We then create a new application by combining the first two. The merged application now references two distinct data sources and effectively creates a (bi-directional) bridge between them. In the final example we combine the third stand alone application - effectively super-imposing the three sets of functionality and data access within one unified application.

Abstract examples are fine, but it’s quite hard to grasp what’s actually going on. So, in the image below we render the three applications more meaningful. We start off with a contact manager. It doesn’t do a great deal apart from allowing us to effectively organize and annotate (generic) contact information. Perhaps it includes modules (components) for communicating with our contacts: email, IM, etc. The second application is a customer relationship management (CRM) system. It’s a lot more specific than the contact manager, but is focused almost entirely on logging and measuring customer engagement. It’s not particularly good at organizing contacts or communicating with them, so it makes a lot of sense to merge the two applications. The operation is not necessarily painless, but owing to the fact that both applications are built from the same basic blocks, a lot of low-level compatibility issues are either eliminated or reduced. Finally, it also makes sense to plug-in the billing system. There’s not much point having a separate system which relies on the same underlying data (contacts). Hopefully, you’ll agree that the resulting application is more useful than the sum of its parts, and, assuming the three independent applications already existed, it will have taken less time to build than a brand-new application. Not many wheels were re-invented.

Perhaps this last example throws up more questions than it actually answers: how is the data merged? Is it actually merged, or is replicated across the three databases? If it’s the latter, does this not give rise to normalization and integrity issues? Can the three stand-alone applications continue to access their original data, or is the combined application incompatible with its ancestors’ data sources? All good questions. Our reply is that the architecture merely facilitates the integration, it doesn’t do it for you. It provides entities such as generic hierarchical data structures (which can contain any arbitrary field format- including references to other data structures). It provides techniques for combining and extending data definitions (which are separate from the data structures themselves). In other words, it provides very general tools (which will be described in detail in later posts) to facilitate this type of integration - but the developer still takes the key design decisions, and ultimately dictates the functionality and information architecture of the new application.

LM-entary, My Dear Watson
Theoretical physics has shown how a small number of fundamental particles can give rise to the hundred or so atoms in the Periodic Table, which in turn give rise to millions of different molecules. It’s strange to think that everything that surrounds us is essentially a specific recombination of the same very small building blocks. By extension, the various interactions between matter (and even life itself) are made possible by the fact that all matter is essentially compatible at the fundamental level. This is a useful, if slightly presumptuous, way of thinking about LM’s micro-component architecture, and it gives us an idea of the diversification which we hope will be possible in the Mixable Web.

In the next post we aim to show this isn’t all theoretical navel-gazing - although there’s been a lot of that too - by going through a real-world (and functioning) example application.

But for the time being, thanks for reading - and if you have any comments be sure to leave them below!

Image Credit

Or why Chrome OS is a missed opportunity

The internet wires have been glowing after Google’s recent Chrome OS announcement. Not much is known about the yet-to-be-released operating system, apart from it being Linux-based, quick booting, and likely to use a modified version of the Chrome browser as its user interface. People are assuming, quite logically, that if the interface is a web browser, the principle aim of Chrome OS is to encourage the use of web applications in place of installed software.

In this post we’ll argue that pushing people towards web applications (with a web-only OS) is an inferior strategy to pulling them in with new and more innovative products. We’ll also argue that the principle drivers for creating better web apps can only come from inside the browser rather than outside. In other words, we’ll argue that Chrome OS represents a missed opportunity.

Some Background Info
If you already know the width of an address bus is not measured in inches, you can probably skip this section, which goes over some basic concepts relating to operating systems.

In the most abstract sense, an operating system just manages a computer’s resources. It allows IO devices such as keyboards, disks and modems to interface with the system. It schedules software to run on the CPU, and it can control which resources are accessible by which programs. The core functionality of operating systems hasn’t evolved much in the 50 odd years they’ve been in existence. They’ve become better – more robust and flexible – but they basically do the same things they have always done.

Most of the obvious innovation has been with user interfaces, which offer a (generally graphics-based) way of interacting with the underlying OS. It’s important to point out that an operating system and its associated UI are distinct components. Some operating systems, such as Microsoft Windows, bundle the core OS and the UI into one package, whereas others, such as Linux, allow different user interfaces to be run on top of a separate OS. This is what Google apparently aims to do with Chrome OS: use the Chrome browser as a graphical interface on top of the core Linux OS.

Why is all this relevant? Well, it becomes very relevant when you start thinking about platforms. Whereas the notions of an OS and a UI are pretty clear, the definition of a platform is much more vague. Depending on the context, an application’s platform can be the operating system, the user interface, or, crucially, a higher-level concept which sits on top of the first two.

Web Applications – Where is the platform?
For web apps, the browser is the platform, not the operating system or user interface. This is the underlying reason why the browser concept has been such a success. The same web page can be viewed by different makes of browser, each running on different operating systems and different hardware. The browser ‘abstracts away’ the differences between how individual systems operate – and presents a common platform upon which web sites and web applications can be built.

So, from the standpoint of innovating web applications, there is not much point in designing an operating system around the browser, because any enhancements will be specific to that platform. It could be argued that if the enhancements were truly useful, they might find their way into other OS/browser combinations – but this is an unlikely scenario, not least because Chrome OS will be one of very few platforms where the browser acts as the front-end to an operating system. Just about everywhere else, the browser is just another installed application, and has no special relationship with the underlying OS.

Innovation comes from Inside the Browser
Improvements in browser technology come from a number of sources. There are the various working parties which suggest improvements to the core technologies used by all browsers (HTML, Javascript, XML, CSS, etc). Then there are the browser makers, who sometimes unilaterally add functionality which is sometimes adopted by the other browsers (innerHTML) - and finally there are the application and framework developers, who create their own scripts which can occasionally become so widely used they get absorbed into the browser itself (JSON parsing, getElementsByClassName).

Out of these three routes, the last one has a key difference. The scripting route to innovation is unique in that it exposes new functionality which is generally available to all browsers immediately. Working party recommendations refer to future implementations - that’s not to say these suggestions are not useful or highly transformative – but it takes time for the browser makers to gradually absorb the suggestions (and not all of them do). A similar situation occurs with unilateral improvements to browsers: until there is widespread cross-browser support, the new functionality is generally not used in large-scale deployments.

The scripting route represents a democratic way of improving browser technology. The most useful functionality rises to the top and is then frequently implemented natively in the browser core.

Pull vs Push
Google has led the way on many fronts in web development, with much of the innovation coming via the scripting route: Gmail demonstrated that it was possible to create a useful browser-based application without using constant page refreshes. And Google Maps blew developers away when it showed what could be achieved by cleverly using Javascript in conjunction with background server calls. These examples helped convince users and developers alike that it would be possible to create truly complex web applications in the future. In other words, they pulled people towards this nascent technology with the promise of functional richness and platform-independent availability. Chrome OS will have no such impact, it pushes people towards the web without actually making the web any better.

It’s ironic that in the blog post announcing Chrome OS, Google actually included the phrase ‘operating systems that browsers run on were designed in an era where there was no web‘. Going back to the previous point that operating systems haven’t really evolved in 50 years, they could have actually said that such systems were designed in an era where there wasn’t much of anything.

We must that remember that the first computer users – IBM’s original customers – were big businesses, governments and research institutions. They used their machines to crunch numbers and query databases full of statistics. This is the era in which operating systems were born, and in which they have basically remained. Software was very specific, and if information had to be shared between programs, a group of dedicated programmers would work out a common file format that would suit both producer and consumer.

Our world is very different. Whilst number crunching applications still exist, both enterprises and individuals are now using their computers principally for communicating and sharing information which has obvious human meaning. The web has made this so much easier, but operating systems still view the world in terms of processes, files and IO streams. Even if a user knows that you can’t put a ‘person’ into a folder containing ‘places’, the operating system doesn’t – it’s just data - a given program may know the difference, but in a truly interoperable world, it’s desirable to have such concepts distinguishable at the platform – not just application – level.

The Fat Lady’s Still in the Limo
If Google really wanted to expand the market for web applications, they would pull users towards that market with enhanced functionality and interoperability, not just providing them with with a free OS, designed for cheap computers, which just gets them rapidly online.

Take mashups - a ugly term to describe a very simple ‘technology’. Some people may find it remarkable that a web page can be divided into various regions which are then populated with information from disparate sources. One can only assume that such people would have gone into fibrillation in the late sixties/early seventies when the cinematic fashion was to divide the screen into sections each showing separate images.

That’s not innovation. Innovation is dragging a user from your Twitter stream (regardless of the client you’re using) and dropping the icon into the address field of a new email, or into a friends folder in Facebook. Innovation is having exactly ONE lists of contacts, which you use for work, friends and family. Innovation is being able to share this list with any application in a structured and selective manner. Innovation is making all of your information available to all of your software tools, within a common semantic framework: no you really can’t drop ‘Bob’ into the ‘my favorite cities’ folder (regardless of what application you’re currently using).

Missed Opportunity
The fact that operating systems deal in primitives which do not fully reflect how we now use computers is really not an issue. The browser is the new the platform, not the OS - and this just means there is not much point trying to add web-focused functionality to a layer that is simply abstracted away by the browser. Google may have other reasons for creating Chrome OS, but improving the web application environment can’t be one of them - or, if it is, it won’t go very far.

Google had (and still has) great scope for pulling people further towards web applications by providing a platform-wide framework for interoperability. So far the web has been great at displaying information – but it’s still pretty poor at allowing users to structure and share information in a generic, selective, and (an at least rudimentary) semantic manner.

We are faced with a great opportunity. Whilst the whole web application ecosystem is still in a pretty primordial state, there’s a window to lay down a well-designed and robust layer of open interoperability. As time passes, this window will gradually close, as various interested parties each create their own schemas, each incompatible with each other, as per the usual script.

If Google really wanted some serious progress in web applications, they should consider creating a general-purpose client-side framework for the interchange of structured and semantically distinguishable information. This is what people need, this is what people want. And if things go well, and such a platform were to gain general acceptance, it might eventually be absorbed into the very fabric of the browser – to the benefit of us all.

Now that would be real innovation.

More on why usage-billing makes more sense than freemium.

A fairly undeniable fact regarding Kamikaze pilots is that they didn’t only remove themselves from the gene pool, they also took out a lot of people who were simply playing by the ‘normal’ rules of war. A similar fate awaits not only those companies hell-bent on distributing their wares for free, but also for many of their more conventional competitors who will get caught by the shrapnel.

Like all good sects, the free web has its own illustrious leader, Chris Anderson, author of the The Long Tail and Free. Anderson has refined his notion of free over time, so that it now no longer focuses primarily on the ad-funded model, preferring instead the trendier freemium concept. His philosophy can be summed up by his phrase “give away the head and charge for the tail”.

The core of the free argument is based on the notion of the distribution cost of digital content being close enough to zero to round down. Hence it doesn’t cost much to give away your product to anyone who wants it, and in doing so you create an opportunity to offer a premium version to your grateful users.

In a review of Anderson’s new book, New Yorker writer Malcom Gladwell points out that even if the distribution cost is very, very small, when you multiply it by a number that is very, very large, for example the number of videos streamed by YouTube, you still get a figure with lots of digits before the decimal point. In other words, if your free offering is really popular, the zero-distribution-cost part of the freemium equation stops being true.

Taking this idea one step further, let’s look at an industry that really does have zero (rather than negligible) distribution costs, and was already doing free when the internet was just a twinkle in TBL’s eye.

Nothing New Under the Sun
In eighties England there were only four TV channels. It is an uncomfortable fact that just about any Englishman in his forties can tell you what triple salco or puissance mean. There were two ad-funded networks, ITV and Channel 4 – the other two channels were provided by the ad-free BBC.

Things must have been pretty sweet for the two ad-funded networks, and even the arrival of two fledgling satellite broadcasters (BSB and Sky) towards the end of the decade did little to upset the status-quo. Most commentators argued that people were very unlikely to pay for a service that could be had for free – and they were almost right. The cost of buying rights to premium content combined with the expenditure required to operate their respective services pushed the two struggling entrants into a shotgun marriage, and British Sky Broadcasting (now known simply as ‘Sky’) was formed.

But then a strange and important thing happened. It appeared the Great British Public was not universally happy with its free programming. It transpired that many people were actually willing to pay for content. Some wanted to see live sporting events, others wanted to see films before they were available to rent, and others just liked the idea of surfing dozens and dozens of channels. Whatever the underlying reasons, the UK rapidly transformed itself into a nation of satellite viewers – as evinced by receiving dishes now nestling like mushrooms on every other house in the UK.

In a move that must surely give the freesters food for thought, an entire nation went from free to paid – and moreover in a industry with a distribution model that represents the best case scenario for free.

Nothing is Identical Under the Sun, Either
Okay, TV in eighties England is not the same as the internet now. Viewers had practically no choice, and the arrival of the satellite operators actually reduced the quality of the free offering, because the satellite operators bought-up the rights to the content people most wanted to watch. Barriers to entry are also very different: it costs a lot less to create a website than a TV network. This last point is important, because when there are only a handful of national operators, there is no real need to do any marketing – everybody knows who you are. One of the key aims of the freemium model is to get your service noticed by letting your free offering do the marketing for you.

Nevertheless the principle still holds: we have seen that under certain circumstances people are willing to pay for content - even in the face of a free alternative. And more to the point: people are willing to pay for the head (the most popular content), even if the (admittedly short) tail can be had for free. This is the exact antithesis of Anderson’s argument.

Any physicist or mathematician can tell you that while you can’t prove a theory with a single example, you can certainly disprove one. So does this imply the free web, and freemium in particular, is based on the incorrect premise that the easier content can be distributed, the less likely people will pay for it? Unfortunately not. Business is not science, and what holds in one market doesn’t necessary hold in another. But we have shown that the pro-free arguments, which at first glance can seem so alluring and self-validating, are in fact nothing of the sort.

That said, the disturbing reality is that, based on current alternatives, the free web – at least in the guise of the freemium model – makes a lot of sense. Or, more succinctly, in the land of blind web revenue models, one-eyed freemium is king.

The real problem, therefore, is not so much with freemium (chronically flawed as it is) - but with the lack of any viable alternatives.

The Paid Web
There are two basic ways of billing for stuff on the web: subscriptions and micro-payments. We have already argued strongly against subscriptions on this blog, and will therefore proffer only a brief summary here: subscriptions, whether on the web or anywhere else, are a form of lock-in. Users know this and so view them with suspicion. People don’t take out subscriptions to restaurants or shops, for example, because they don’t want to limit their future choices. Most importantly, they resent paying for something they either no longer use, or use less than expected.

Subscriptions can work when people are willing to exchange a degree of product choice for some sort of benefit (most often a discount). This can occur with magazines or newspapers, when the customer realizes they would have probably bought the publication anyway. But in the fast-moving world of the web, where people don’t know what’s coming next and so don’t want to limit their choices, and where there’s so much free stuff available anyway, subscriptions (even in the context of freemium conversion) don’t work.

As to micro-payments - the practice of charging a tiny sum for each incremental bit of service or content - Clay Shirky killed that idea long ago. He quite rightly pointed out that regardless of the size for the requested payment, the ‘mental transaction costs’ associated with an endless stream of tiny decisions rapidly add up to one big and annoying cost in terms of time (our most valuable resource) and inconvenience. Micro-payments are even worse than subscriptions.

A Thought Experiment
Einstein loved thought experiments (and look where they got him) – so let’s try one here. In a way it’s ironic that we’ve just held up newspapers and magazines as an example of how subscriptions can work in the real world, when these very same two categories are getting absolutely hammered on the web. So let’s balance the books and use newspapers as the basis for our though experiment.

Let us imagine a magical world in which a copy of a newspaper can be printed and distributed for free to every household in the world. Let us also imagine that each issue has a special microchip embedded inside. Many people will pick up their free paper every morning and throw it straight in the trash. Others may skim the front and back pages over breakfast and then throw it in the trash. But a certain percentage (dependent mainly on how good the newspaper is) will actually open it up and read inside. This is were the microchip comes in – it’s clever enough to tell when the newspaper is being read properly.

After an acceptable period of product engagement, it beeps and says, “Good morning, I can see you’re really getting into the paper today. How about we charge you ¢20 every time you read more than half a dozen articles?”. The reader has a little think and asks, “What if one day I don’t read the paper, or only read four articles?”. “Then we won’t charge you”, chirps the microchip. Our reader likes what he’s hearing, but he thinks a bit longer and then adds “Clay Shirky says I’ll rapidly get fed up deciding every morning whether I want to be charged for reading the paper”. The microchip was expecting this one: “Clay’s right. We only ask you this once whether you are willing to pay for reading the paper thoroughly. If you agree, you’ll just get a monthly bill, based on how much you’ve read”. “What if I find a better newspaper and stop reading this one?” asks the reader. “Then you’ll never hear from us again” says the chip.

Our reader puts the paper down and begins to reason: what’s my downside? I can’t buy a good newspaper for ¢20, and if I don’t read it, I don’t pay. No annoying interruptions. No sections I don’t have access to. I only pay for what I consume, and if I stop reading it altogether, it just disappears. Finally he exclaims, “Microchip dude, you gotcha self a deal!”.

Fantasy, eh? Not entirely. What a lot of people don’t seem to grasp, or at least leverage, is the fact that the web is bi-directional. This model can’t work for TV or satellite because they are broadcast mediums - the information only flows one way. But it can and will work on the net.

Not Quite Cloud Billing
We call this model micro-billing. One decision, one bill, very small payments.

In many ways it’s similar to the way billing works in the cloud, in that you only pay for what you use. But there are key differences. In the cloud, billing generally starts the moment you begin using the service, but micro billing uses a usage threshold, below which you don’t pay. It’s very important that people are able to try products without paying – this is the best part of the freemium model. It makes sense for content owners to leverage the web’s negligible distribution costs to get their products in front of consumers. The product functions as its own marketing device.

Micro-billing also has a maximum usage fee. In our thought experiment the threshold payment and the maximum payment were the same (¢20), but in some situations it makes sense for them to be different. The key point is that a potential customer should always be able to work out the most the service will cost. People hate getting into open-ended commitments.

Finally, a word on price. How can a quality newspaper only cost ¢20? Well, for the same reason you can find electronic gadgets in cereal packets: volume. With over one billion people connected to the web, companies with content that can be digitized have to learn to offset the loss of their semi-monopolistic status in the physical world, with access to this much larger, albeit more competitive, online market. In a key difference with the freemium model, all active users pay. Some may pay more than others – but they all pay. When you have a very large customer base and such low variable costs, you can charge much less for your product and still maintain a decent return on capital.

In the freemium model, on average 97% of people use a service without paying for it. This means the price for the paying users is much higher than it could/should be because 3% of the user base is effectively subsidizing the remainder. Micro-billing significantly lowers the cost of a service, increasing the likelihood people will regard it as good value, and hence sign-up for being billed

Sayonara Baby
Free to use (as opposed to try) is a dangerous trend, not least because it also punishes those current and potential competitors who can see it doesn’t make sense. Tim Cohn put it best when he asked “is free the new barrier to entry”.

Fortunately, the example of the TV market in England shows that all is not lost – it is possible to introduce a paid model into a market which was previously all free. But the model has to be right. Currently, there is no compelling paid model for the web, and so freemium wins by default.

We believe usage-based micro-billing is the paid model the web has been waiting for.

Photo Credit

Or why usage billing makes more sense

In the freemium model, a free version of a product is used as a marketing tool to entice people to sign-up for an extended paid version. The term was first coined on Fred Wilson’s blog back in 2006.

Freemium has a great deal of support on the net, especially after advertising rates plummeted, and the practice of buying-out companies based purely on their reach has gone the way of the codpiece. These days investors and potential company buyers want to hear the sweet jangle of coins dropping into the coin box. Well-known services like Flickr and Skype, as well as many others, use the freemium model.

Close, but no cigar
In an excellent article, Free! Why $0.00 Is the Future of Business, Chris Anderson, author of The Long Tail and soon-to-be-released Free, explains why this “freaky land of the free” makes perfect sense – at least to him. He begins by pointing out that the concept of giving away something in the hope of charging for something else is anything but new, describing Gillette’s groundbreaking idea of giving away razors and then charging for blades. He also cites the example of the Brazilian Banda Calypso, who intentionally make their music and sleeve art available online, so that street vendors can create and sell cheap CDs, legally keeping all the money. The buzz created around the CDs permits Banda Calypso to sell-out its live concerts.

Banda Calypso’s business model is great. It leverages the negligible distribution cost of digital content to get its promotional material (the music) in the hands of an army of marketeers, who are happy take on the production costs, and plaster the band’s image all over the place in order to promote both their own and the band’s interests. It’s beautiful - but it’s not freemium.

As Anderson points out, both are examples of a general form of cross-subsidy, were one type of product is given away so as to charge for another. Neither Gillette’s original strategy, nor Banda Calypso’s, meet the common definition of freemium as it exists on the web, which dictates that both the free and premium versions are variations of the same product. To be genuinely freemium, Gillette would have to offer a two blade head for free, and charge for a four-bladed one; Calypso would have to give away most of their concert seats, and only charge for the best ones.

The pitch
The logic behind freemium is that for many websites the marginal cost of providing the service is very low. And as hardware and cloud services continue to get cheaper, it gets lower by the day. Take advantage of this to create a large base of non-paying users, then put your faith in the law of averages and watch some of them convert to the paid service.

Even if the number of paying users is only a tiny fraction of the freeloaders, that’s still more users than you would have got by just offering a straight subscription service, and their acquisition cost (ie the cost of running the free service) is significantly lower than a large-scale advertising and referral program.

Very nice. So much so that Sean O’Malley was driven to ask whether freemium can save web 2.0? - without, it’s worth adding, suggesting that it could.

Freemium is not a revenue model - it’s a marketing technique. The revenue model is still subscription based. The core idea is that giving away a product version is a more cost-effective way of driving subscriptions than traditional methods. And it probably is. But let’s take a closer look at the opportunity cost of letting anybody and everybody use your service for free.

Under the hood
There are hundreds of web pages discussing freemium. Based on nothing more sophisticated than a simple frequency count within many of these pages, an estimate of the average conversion rate for freemium comes in at around 3%. It’s true that some verticals have higher rates (5-10%) but it’s equally true that others have much lower ones (0.5 – 0.05%).

So let’s assume that a hypothetical and profitable website is using freemium, and charges $10/month for its paid service. This means the company is earning an average 30¢ from all active users. Let’s also assume the core product is a spreadsheet application, since it’s a common metaphor.

Okay. So you’re a happy and contented user of the free version. You think it’s really cool that you can use such a fantastic program at no cost. Then one day you need to save your spreadsheet in CSV format. No problem, just click on the ’save as CSV’ button. You guessed it: “The save as CSV option is only available to premium users, click here to upgrade”. This kind of message is an extraordinarily effective way of making even the most mild-mannered person shout obscenities. So what do you do? Pay $10 to save your file as a CSV? Perhaps you do. Perhaps…

The website owner knows these messages are annoying and can frequently be counterproductive. There are number of ways of minimizing them. The first is to make the CSV option available in the free version too. But you don’t have to be Warren Buffet to realize this approach doesn’t scale. If all the useful functionality is free, why would anyone upgrade to the premium version? (A whole micro industry has cropped-up offering advice on how to optimize the free/paid balance). A more cunning technique is to not show the CSV button in the free version. If you can’t click on it, you can’t get pissed off when it doesn’t work. But now there’s a new problem: if you can’t see what your missing, why would you ever upgrade?

Enter the sneak preview. At regular intervals, free users are given access to some premium features. They can now see what they’re missing, and might even be persuaded into hitting the mythical upgrade button when the functionality disappears at the end of the promotion. Very clever. But let’s dig a little deeper…

The free version of any freemium service, must, by definition, be useful – otherwise even the freeloaders wouldn’t use it. You can’t give them ‘copy’ and hide ‘paste’ behind the pay wall. So what goes into the premium version? Functionality which is occasionally very handy, but not fundamental to core usage. Like a save as CSV option. And what are free users expected to think when they are given a time-limited opportunity to use these nice-but-non-core-features? Yay! For a limited time I can save all my spreadsheets as CSVs! Game changer! Not. Like your nail clippers, you only miss this kind of functionality when you need but can’t find it.

All this assumes the website owner is in control of what’s in the different versions of the product. But if your spreadsheet competitor offers the CSV option in their free version then you probably will too. If you don’t and it doesn’t hurt you, then perhaps people aren’t saving that many spreadsheets in CSV format, and so why would they pay for that functionality anyway?

What we have here is the risk of a glorious race to the bottom. Your competitors will offer more and more free functionality so as to take market share. All this is great for your users, but not so good for your shareholders.

Setting aside the fact that all the time dedicated to deciding what should be free and what shouldn’t - often backed-up by complex A-B testing – could be better employed by improving the core product. Or that your competitors are ruining the market by dumping evermore free stuff on grateful users, there is one more reason why freemium is not the savior it’s frequently held up to be - and it’s a humdinger.

The Deal-Breaker
Defenders of the freemium model could point out the spreadsheet example is somewhat contrived, or that it focuses solely on functionality. A common freemium approach is to offer full functionality in both versions, and to simply offer more of something in the paid version. More storage, more uploads, more downloads.

Fair enough. But here’s one thing freemium fans can’t deny: in their model, a tiny minority of paid users subsidizes the service for everybody. It is this simple fact that makes the freemium model self-defeating, because, for the numbers to work, the price of the paid service must be set artificially high.

In other words, for the marketing side of the freemium model to work, the free offering must be functional and useful, otherwise no-one would use it. The premium version, therefore, can only contain an incremental benefit. But the difference between free and $10, or $5, or even $1, is anything but incremental.

The key flaw in the premium model is that it requires people to overpay for the inherently incremental advantage of the premium service. It has to do this, because the premium users must support the whole user base.

As we saw above, the spreadsheet service could, in theory, be profitably supplied to the entire user base at 30¢ per user per month. But there is a problem. Even if the conversion rate would be expected to rise steeply, it would not reach 100% since some people just refuse to pay, at any price. Josh Kopelman calls this resistance the Penny Gap.

The Penny Gap and the Dollar Trap
In his famous blog post, Kopelman explains why the normal price/demand relationship breaks down on the net when the price hits zero. He points out that it’s significantly harder to get someone to go from $0 to $1 than it is to get them to go from $1 to $2 - even if the gap is the same. The penny gap offers indirect support to the freemium model because it says that no matter how low the price is, some users just won’t cough up. On that basis, it’s better to get a tiny minority of users to effectively pay for everyone else.

There’s no doubt the penny gap exists. So the obvious question is: why?

Does Amazon suffer from the penny gap? It doesn’t look like it. People seem more than happy to pay for the convenience of having their stuff delivered to their door. So what is it about certain e-commerce services that allow them to convince people to open their wallets?

One argument is that people are more willing to pay for tangible goods. Those same goods they can see for sale in the shops, which come with a price tag attached, and are rarely given away for free. The implication is that the free web has made a rod for its own back: by giving away so much stuff, the perceived value of that stuff has fallen to zero, thus making it very hard to get large numbers of people to buy any type of web service at any price. This is an intriguing idea, and worthy of a blog post in its own right. But for today at least, save for pointing out that the Apple App Store doesn’t seem to have this problem, we’re not going to go there.

Another possible explanation (also supported by the App Store) is that there is a fundamental difference between a one-off purchase and a subscription. The recurring nature of subscriptions are exactly what make them so attractive to companies – but users aren’t stupid. They can add up. A monthly subscription measured in pennies will add up to a cost measured in dollars over time, regardless of how often the service is actually used.

There is also the fact that frequently companies make it very difficult to get out of subscriptions, using minimum terms and sometimes invoking penalty clauses for cessation. Even if un-subscribing from your service is only a mouse-click away, the whole concept may have been so tarred in the minds of consumers they don’t even give you the chance to explain how easy it is to drop your service.

There’s also the question of convenience. People don’t like typing their credit card details into payment systems, even if they’re not worried about fraud. It’s a pain and the work involved is the same regardless of whether you’re about to pay $30 or 30¢ – in fact, it seems almost stupid to pay 30¢ with a credit card. Amazon cracked this one right away when they created a system to store your details online. After the initial purchase, the subsequent ones are practically painless.

So, while acknowledging the Penny Gap exists, the reason for it existing – at least for subscription services – could have less to do with the gap between $0 and $1 and more to do with the very nature of subscriptions: the risk of lock-in, the fact that you still pay even if you don’t use the service, the inconvenience (and potential risk) of having to initiate each subscription with a separate credit card transaction.

Assuming we’re on the right lines then, and the problem is not with paying for web services per se, rather with all the baggage associated with subscriptions – what’s to be done? What’s the solution?

MicroBilling
Wikipedia defines microbilling as a service for mobile phones. Here we’ll use a different definition: microbilling involves charging usage-based fees for the use of web services. These micro payments across multiple websites are consolidated into one single monthly credit card or bank debit transaction.

Usage-based billing has been tried before, in a different context to web services, without much success. Micro-payments, at least for content, are pretty much discredited. Why bother even suggesting microbilling for web services has got a chance?

Because it solves all the problems associated with subscriptions. Because it can be made to work. Because it is elegant.

Let’s assume one or more clearing houses exist for handling microbilling. Users create an account, enter their payment details once and then they’re all set up. They find out about participating websites in the same way as they find out about freemium websites: virality, referrals, the news.

They chance upon a new spreadsheet service – it looks interesting. They create an account as normal, perhaps using their credentials from the microbilling service. They can try out the service for free. There is none of this free/premium nonsense – they have access to full functionality from day one.

The microbilling system has minimum and maximum usage thresholds. If the user starts using the service on a regular basis, sooner or later they’ll hit the minimum threshold. A message pops up, explaining that the threshold has been reached, and that if the user wants to continue using the service the minimum threshold payment is 25¢ and the maximum is 50¢ per month. There is a one-click button to agree to any (potential) billing. The user doesn’t need to be Fields Medal winner to work out that the most this service can cost per year is $6. The user clicks yes and goes back to using the product. The next month he goes on holiday and doesn’t use the service much. How much is he billed? Zero. The month after that he has a lot to catch-up on and uses it every day. Monthly bill: 50¢.

Perhaps in a years time, he finds a better spreadsheet app, and stops using the service. How does he un-subscribe? He doesn’t have to. He never subscribed in the first place. By simply not using the system, he won’t ever get billed for it again. He likes microbilling so much he explains it to his mother. She gets it the first time round.

He can log in to the microbilling system at any time and see the status of his current usage for each website, and also how close it is to the minimum and maximum thresholds. Once a month he gets debited for his consolidated usage across all his microbilling websites. He can see a highly itemized bill online, which gives the breakdown of the single transaction he sees on his credit card or bank statement. He never has to swear at a ‘do you want to upgrade?’ message again.

In a nutshell, by making the payment process fairer, painless and transparent, the Penny Gap can be bridged. With a greatly reduced Penny Gap, the base of paying users will expand significantly. With lots of paying users, websites can begin offering their services at a much fairer price because nobody is subsidizing anybody else. With such low prices for services, more people will sign-up, further expanding the user base and further driving down the cost of the service. Rinse and repeat.

Application or service providers can focus their time on innovating their core products, instead of continually testing different variations of the free/premium mix, or deciding what functionality should be included in this week’s sneak preview. This extra time will come in very handy because their users are no longer locked-in with subscriptions. If the website doesn’t continually improve, the users will migrate to greener (though, at these prices, not necessarily cheaper) pastures.

Website owners will wake up to the fact that frequently you can make more by billing a lot a people for a small amount, rather than billing a small number a large amount.

If microbilling can reach critical mass, there will be no going back.

David Semeria
LM Framework
June 2009

Disclosure
LM Framework will use microbilling through the P-BOS licensing system. This post basically summarizes the process we went through when we evaluated the freemium model internally, and as a consequence decided to create our own usage-based revenue model.

This new video demonstrates some of the wider concepts behind LM.

It focuses on the ability to draw in information from different sources; add structure and metadata to the information, and then share it (either by using shared folders, or in the specific case of Twiggler, via Twitter).

The most important point is that it is really easy to set up folders in LM for lots of different types of information. For example: Flash videos, audio files, bookmarks, online documents etc.

We’ll be adding more functionality to Twiggler in the coming months, and we’ll post new videos as we go along.

Also: we’d love to hear you feedback, thanks.

.

Twiggler is the first true application created using LM Framework. If you’ve seen the demo video, we would love to hear your feedback.

.

LM Framework is a very flexible system for specifying and delivering web applications. It allows developers to create the type of open and powerful web applications users will want in the future.

Feel free to comment on any aspect of the system.

.

P-BOS is a new licensing system for open source software. It allows open source developers to earn revenues from their work, if they want to.

If you have any views on P-BOS, you can share them here.

.