Just because we use certain tools very frequently, doesn’t necessarily mean we understand them as well as we could. Often, we use patterns we know will work, without really understanding why they work. This is certainly the way I use some of the more obscure parts of Javascript.

And so, after finding myself with some free time, I decided to sit down and think more deeply about a topic I generally try not to think too hard about. The topic I chose was the relationship (which previously I merely suspected to exist) between scope, garbage collection and closures in Javascript.

The results of my ruminations are collated in the presentation embedded here. My understanding of this area is now much clearer than it was before. I published the presentation in the hope that other people might find it useful too.

Needless to say, it’s easier to read in full-screen mode.

Zooming In
This is the second in a series of articles on the LM micro-component architecture. Click here to start from the beginning.

In this post we’re going to describe components in more detail, and use a real application to show the micro-component architecture in action. For the sake of clarity, we’ve simplified some of the details, but the underlying model as described here is essentially identical to that of the functioning framework.

Within LM, components are either atomic in nature, or else they are composed of other components. Atomic components can either be concrete (HTML, CSS, JS or content) or else they are abstract. More on abstract components later.

The image below shows a typical component breakdown for an interface button. It can be seen that the button is a composite component, and, even though it represents a very simple interface item, it is itself composed of quite a high number of sub-components. At first glance, it may seem like overkill to break something as simple as a button down into so many parts….

… but if we consider (image below) a component representing a button group, we can immediately see the low-level factoring that occurs naturally. It should be clear that both graphical and behavioral consistency are reinforced by explicitly specifying the component breakdown in such detail.

A Real-World Example
We can see that the images below describe two folder managers which share the same general structure. At first glance, they appear more alike than they actually are. Each folder manager deals with a different underlying data type (images or contacts) and behaves differently depending on the context (double clicking an image fires-up the image viewer, whereas double clicking a contact brings up the contact properties form). That said, they also have a great deal in common. Apart from the graphical layout, many behaviors are also shared: dragging and dropping has the same effect in both contexts, as do the cut, copy and paste buttons, for example.

The image below shows a breakdown of some graphical components in the generic folder manager. The folder manager also references many other types of components, but it’s harder to visualize them graphically. The highlighted parts represent the various button groups (as described above) and the various icon templates. We have already seen that button groups share a wide range of common components. Similarly, behind the scenes, both folder managers share many items. For example, the same drag and drop library is used by both instances. Combined with the fact that they both use the same type of data structure to represent two quite different hierarchies, this means both instances are highly compatible with each other. Although it doesn’t make much sense to drop an image into a contact folder, if both folders contained similar data types, it would be straightforward to drag and drop items between the two windows (with both source and destination data structures being automatically updated both in the bowser and on the server).

E Pur Si Muove
And just to show that these examples do indeed come from a working application, below is a two minute video which shows the contact and image managers being used within a Twitter client which is written in LM.

It’s all about Structure
By now, it should be clear that intense factoring is one of the defining principles behind the micro-component architecture. And, although the LM compiler does carry-out some additional factoring, the key point is that the architecture itself promotes the natural identification of recurring structure at the specification level.

The first post emphasized the high level advantages of this granular component sharing: increased inter- and intra- application compatibility, leading to the possibility of complex application ecosystems emerging, in which applications can be combined, re-combined and augmented with ease.

In the next post, we’ll look at some low-level advantages of the micro-component architecture. We’ll see how it allows highly complex applications (comparable to desktop software) to be delivered to the browser in a highly efficient manner, even over high latency (eg mobile) networks.

The Big Picture
LM Framework uses a software architecture in which web applications and services are designed and delivered as a series of generic building blocks, which we call micro-components. This is the first in a series of posts which together will describe the architecture, explain the rationale behind many of the design decisions, and point-out why we’re excited about its potential. The series will start off at a high level, and will become increasingly more detailed and implementation orientated. In this post we’ll introduce the Mixable Web - we’ll explain what it is, and how it could possibly influence the way we build and use internet software and services in the future.

The Mixable Web is simply an alternative way of viewing software and data, and is very general. As such, it applies both to how users interact with applications, and how applications themselves are constructed. It hinges on the notion of compatibility. The logic behind the Mixable Web is very straightforward: the more applications (or more precisely the components from which they are built) are compatible with each other, the richer the user experience, and the more power in the hands of the developer.

Size Really Does Matter
The image below shows a stylized component breakdown for an application. It’s not necessary at this stage to know what the application does, or what the components represent. The key point is that a high proportion of components are referenced more than once within the specification. This is what facilitates the underlying compatibility between the various parts of the application, and, as we’ll see in a moment, it will also enable us to create new applications by simply combining existing ones.

The skeptical reader is probably thinking “So what? There’s nothing new here”. And whilst it’s true that component re-use is an established practice in software engineering, it’s also rare to see the concept taken to such extremes (in LM, components can be as small as a single image, or a line of text). In future posts we’ll list some low-level advantages of the micro-component model (which are closely linked to the current limitations of both the browser, and of delivering a web application as a series of distinct web pages). But for the time being, we want to focus on the high-level issues concerning deep component compatibility - and the resulting benefits to both the user and the developer.

In the image below we can see what happens when two applications are combined. It can be seen the two stand-alone applications share many components. This is not a coincidence. The smaller and more general purpose the building blocks, the more likely they can be re-used in different contexts, which leads to a more granular compatibility both within and between applications. This also explains why the original multi-purpose Lego blocks are much more versatile than today’s highly-specific ones!

We’ll be seeing this image again in a later post where we talk about compression and latency - two key issues for web-hosted software (SaaS). There are many other reasons for being happy to see so many shared components in the new combined application. Setting aside the footprint issues mentioned above, here are some of them:

• Graphical consistency
  The combined application shouldn’t look like a patchwork quilt
• Behavioral consistency
  Clicking on the same button in different contexts has the same effect
• Customization consistency
  The developer (and in theory the user) can customize components.
• Data consistency
  It should be possible to share data between the two sub-applications
• Easier development
  Using existing components represents less work than creating new ones
• Less testing
  Existing components are more likely to be stable

The fourth point, data consistency, has major ramifications. One of the key differences between desktop and web applications is their relative levels of interconnectivity. Desktop applications are more like islands: information can travel between them, but the journey is often not easy. Web applications, on the other hand, frequently have data integration at their core - this a direct consequence of the web’s open and multi-point architecture. Much has been made of ‘mashups’, where data from diverse sources is presented on the same web page, but this still represents a mainly read-only situation. In the future, it will be possible not only to read, but also share, organize and integrate information from disparate sources, all from within the same application.

Application Ecosystems
Just about every web site connects to a some sort of database. Most of the time, the only way to access the database is via the website, although some services now make a distinction between the two (Twitter is a good example). The micro-component architecture allows data sources to be viewed as components themselves (each with its own access permissions). And since LM applications share so much low-level DNA, it will be possible to seamlessly integrate information from different sources within the same application. This means going beyond simply displaying the information on the screen, and transforms applications into something closer to mini-platforms - with the difference that they do not necessarily have to remain isolated. To return to a geographical analogy: an application can start off as a small island, which is then gradually integrated with other applications to form ever larger and more complex land masses: bigger islands, continents, and - why not? - even planets!

This potential for combining and re-combining functionality and data sources into ever richer (or more focused) applications is exciting. There are some pretty clear evolutionary analogies, which open-up the possibility of dense application ecosystems emerging. But let’s not get ahead of ourselves. To flesh-out these concepts, we’ll present two examples; one generic and one specific.

In the image below we can see three stand alone applications. For simplicity we have each application referencing only one data source, but there is no reason why they should not reference more. We then create a new application by combining the first two. The merged application now references two distinct data sources and effectively creates a (bi-directional) bridge between them. In the final example we combine the third stand alone application - effectively super-imposing the three sets of functionality and data access within one unified application.

Abstract examples are fine, but it’s quite hard to grasp what’s actually going on. So, in the image below we render the three applications more meaningful. We start off with a contact manager. It doesn’t do a great deal apart from allowing us to effectively organize and annotate (generic) contact information. Perhaps it includes modules (components) for communicating with our contacts: email, IM, etc. The second application is a customer relationship management (CRM) system. It’s a lot more specific than the contact manager, but is focused almost entirely on logging and measuring customer engagement. It’s not particularly good at organizing contacts or communicating with them, so it makes a lot of sense to merge the two applications. The operation is not necessarily painless, but owing to the fact that both applications are built from the same basic blocks, a lot of low-level compatibility issues are either eliminated or reduced. Finally, it also makes sense to plug-in the billing system. There’s not much point having a separate system which relies on the same underlying data (contacts). Hopefully, you’ll agree that the resulting application is more useful than the sum of its parts, and, assuming the three independent applications already existed, it will have taken less time to build than a brand-new application. Not many wheels were re-invented.

Perhaps this last example throws up more questions than it actually answers: how is the data merged? Is it actually merged, or is replicated across the three databases? If it’s the latter, does this not give rise to normalization and integrity issues? Can the three stand-alone applications continue to access their original data, or is the combined application incompatible with its ancestors’ data sources? All good questions. Our reply is that the architecture merely facilitates the integration, it doesn’t do it for you. It provides entities such as generic hierarchical data structures (which can contain any arbitrary field format- including references to other data structures). It provides techniques for combining and extending data definitions (which are separate from the data structures themselves). In other words, it provides very general tools (which will be described in detail in later posts) to facilitate this type of integration - but the developer still takes the key design decisions, and ultimately dictates the functionality and information architecture of the new application.

LM-entary, My Dear Watson
Theoretical physics has shown how a small number of fundamental particles can give rise to the hundred or so atoms in the Periodic Table, which in turn give rise to millions of different molecules. It’s strange to think that everything that surrounds us is essentially a specific recombination of the same very small building blocks. By extension, the various interactions between matter (and even life itself) are made possible by the fact that all matter is essentially compatible at the fundamental level. This is a useful, if slightly presumptuous, way of thinking about LM’s micro-component architecture, and it gives us an idea of the diversification which we hope will be possible in the Mixable Web.

In the next post we aim to show this isn’t all theoretical navel-gazing - although there’s been a lot of that too - by going through a real-world (and functioning) example application.

But for the time being, thanks for reading - and if you have any comments be sure to leave them below!

Image Credit

Or why Chrome OS is a missed opportunity

The internet wires have been glowing after Google’s recent Chrome OS announcement. Not much is known about the yet-to-be-released operating system, apart from it being Linux-based, quick booting, and likely to use a modified version of the Chrome browser as its user interface. People are assuming, quite logically, that if the interface is a web browser, the principle aim of Chrome OS is to encourage the use of web applications in place of installed software.

In this post we’ll argue that pushing people towards web applications (with a web-only OS) is an inferior strategy to pulling them in with new and more innovative products. We’ll also argue that the principle drivers for creating better web apps can only come from inside the browser rather than outside. In other words, we’ll argue that Chrome OS represents a missed opportunity.

Some Background Info
If you already know the width of an address bus is not measured in inches, you can probably skip this section, which goes over some basic concepts relating to operating systems.

In the most abstract sense, an operating system just manages a computer’s resources. It allows IO devices such as keyboards, disks and modems to interface with the system. It schedules software to run on the CPU, and it can control which resources are accessible by which programs. The core functionality of operating systems hasn’t evolved much in the 50 odd years they’ve been in existence. They’ve become better – more robust and flexible – but they basically do the same things they have always done.

Most of the obvious innovation has been with user interfaces, which offer a (generally graphics-based) way of interacting with the underlying OS. It’s important to point out that an operating system and its associated UI are distinct components. Some operating systems, such as Microsoft Windows, bundle the core OS and the UI into one package, whereas others, such as Linux, allow different user interfaces to be run on top of a separate OS. This is what Google apparently aims to do with Chrome OS: use the Chrome browser as a graphical interface on top of the core Linux OS.

Why is all this relevant? Well, it becomes very relevant when you start thinking about platforms. Whereas the notions of an OS and a UI are pretty clear, the definition of a platform is much more vague. Depending on the context, an application’s platform can be the operating system, the user interface, or, crucially, a higher-level concept which sits on top of the first two.

Web Applications – Where is the platform?
For web apps, the browser is the platform, not the operating system or user interface. This is the underlying reason why the browser concept has been such a success. The same web page can be viewed by different makes of browser, each running on different operating systems and different hardware. The browser ‘abstracts away’ the differences between how individual systems operate – and presents a common platform upon which web sites and web applications can be built.

So, from the standpoint of innovating web applications, there is not much point in designing an operating system around the browser, because any enhancements will be specific to that platform. It could be argued that if the enhancements were truly useful, they might find their way into other OS/browser combinations – but this is an unlikely scenario, not least because Chrome OS will be one of very few platforms where the browser acts as the front-end to an operating system. Just about everywhere else, the browser is just another installed application, and has no special relationship with the underlying OS.

Innovation comes from Inside the Browser
Improvements in browser technology come from a number of sources. There are the various working parties which suggest improvements to the core technologies used by all browsers (HTML, Javascript, XML, CSS, etc). Then there are the browser makers, who sometimes unilaterally add functionality which is sometimes adopted by the other browsers (innerHTML) - and finally there are the application and framework developers, who create their own scripts which can occasionally become so widely used they get absorbed into the browser itself (JSON parsing, getElementsByClassName).

Out of these three routes, the last one has a key difference. The scripting route to innovation is unique in that it exposes new functionality which is generally available to all browsers immediately. Working party recommendations refer to future implementations - that’s not to say these suggestions are not useful or highly transformative – but it takes time for the browser makers to gradually absorb the suggestions (and not all of them do). A similar situation occurs with unilateral improvements to browsers: until there is widespread cross-browser support, the new functionality is generally not used in large-scale deployments.

The scripting route represents a democratic way of improving browser technology. The most useful functionality rises to the top and is then frequently implemented natively in the browser core.

Pull vs Push
Google has led the way on many fronts in web development, with much of the innovation coming via the scripting route: Gmail demonstrated that it was possible to create a useful browser-based application without using constant page refreshes. And Google Maps blew developers away when it showed what could be achieved by cleverly using Javascript in conjunction with background server calls. These examples helped convince users and developers alike that it would be possible to create truly complex web applications in the future. In other words, they pulled people towards this nascent technology with the promise of functional richness and platform-independent availability. Chrome OS will have no such impact, it pushes people towards the web without actually making the web any better.

It’s ironic that in the blog post announcing Chrome OS, Google actually included the phrase ‘operating systems that browsers run on were designed in an era where there was no web‘. Going back to the previous point that operating systems haven’t really evolved in 50 years, they could have actually said that such systems were designed in an era where there wasn’t much of anything.

We must that remember that the first computer users – IBM’s original customers – were big businesses, governments and research institutions. They used their machines to crunch numbers and query databases full of statistics. This is the era in which operating systems were born, and in which they have basically remained. Software was very specific, and if information had to be shared between programs, a group of dedicated programmers would work out a common file format that would suit both producer and consumer.

Our world is very different. Whilst number crunching applications still exist, both enterprises and individuals are now using their computers principally for communicating and sharing information which has obvious human meaning. The web has made this so much easier, but operating systems still view the world in terms of processes, files and IO streams. Even if a user knows that you can’t put a ‘person’ into a folder containing ‘places’, the operating system doesn’t – it’s just data - a given program may know the difference, but in a truly interoperable world, it’s desirable to have such concepts distinguishable at the platform – not just application – level.

The Fat Lady’s Still in the Limo
If Google really wanted to expand the market for web applications, they would pull users towards that market with enhanced functionality and interoperability, not just providing them with with a free OS, designed for cheap computers, which just gets them rapidly online.

Take mashups - a ugly term to describe a very simple ‘technology’. Some people may find it remarkable that a web page can be divided into various regions which are then populated with information from disparate sources. One can only assume that such people would have gone into fibrillation in the late sixties/early seventies when the cinematic fashion was to divide the screen into sections each showing separate images.

That’s not innovation. Innovation is dragging a user from your Twitter stream (regardless of the client you’re using) and dropping the icon into the address field of a new email, or into a friends folder in Facebook. Innovation is having exactly ONE lists of contacts, which you use for work, friends and family. Innovation is being able to share this list with any application in a structured and selective manner. Innovation is making all of your information available to all of your software tools, within a common semantic framework: no you really can’t drop ‘Bob’ into the ‘my favorite cities’ folder (regardless of what application you’re currently using).

Missed Opportunity
The fact that operating systems deal in primitives which do not fully reflect how we now use computers is really not an issue. The browser is the new the platform, not the OS - and this just means there is not much point trying to add web-focused functionality to a layer that is simply abstracted away by the browser. Google may have other reasons for creating Chrome OS, but improving the web application environment can’t be one of them - or, if it is, it won’t go very far.

Google had (and still has) great scope for pulling people further towards web applications by providing a platform-wide framework for interoperability. So far the web has been great at displaying information – but it’s still pretty poor at allowing users to structure and share information in a generic, selective, and (an at least rudimentary) semantic manner.

We are faced with a great opportunity. Whilst the whole web application ecosystem is still in a pretty primordial state, there’s a window to lay down a well-designed and robust layer of open interoperability. As time passes, this window will gradually close, as various interested parties each create their own schemas, each incompatible with each other, as per the usual script.

If Google really wanted some serious progress in web applications, they should consider creating a general-purpose client-side framework for the interchange of structured and semantically distinguishable information. This is what people need, this is what people want. And if things go well, and such a platform were to gain general acceptance, it might eventually be absorbed into the very fabric of the browser – to the benefit of us all.

Now that would be real innovation.