The Mixable Web (1)

The Big Picture
LM Framework uses a software architecture in which web applications and services are designed and delivered as a series of generic building blocks, which we call micro-components. This is the first in a series of posts which together will describe the architecture, explain the rationale behind many of the design decisions, and point-out why we’re excited about its potential. The series will start off at a high level, and will become increasingly more detailed and implementation orientated. In this post we’ll introduce the Mixable Web - we’ll explain what it is, and how it could possibly influence the way we build and use internet software and services in the future.

The Mixable Web is simply an alternative way of viewing software and data, and is very general. As such, it applies both to how users interact with applications, and how applications themselves are constructed. It hinges on the notion of compatibility. The logic behind the Mixable Web is very straightforward: the more applications (or more precisely the components from which they are built) are compatible with each other, the richer the user experience, and the more power in the hands of the developer.

Size Really Does Matter
The image below shows a stylized component breakdown for an application. It’s not necessary at this stage to know what the application does, or what the components represent. The key point is that a high proportion of components are referenced more than once within the specification. This is what facilitates the underlying compatibility between the various parts of the application, and, as we’ll see in a moment, it will also enable us to create new applications by simply combining existing ones.

app 1

The skeptical reader is probably thinking “So what? There’s nothing new here”. And whilst it’s true that component re-use is an established practice in software engineering, it’s also rare to see the concept taken to such extremes (in LM, components can be as small as a single image, or a line of text). In future posts we’ll list some low-level advantages of the micro-component model (which are closely linked to the current limitations of both the browser, and of delivering a web application as a series of distinct web pages). But for the time being, we want to focus on the high-level issues concerning deep component compatibility - and the resulting benefits to both the user and the developer.

app 2

In the image below we can see what happens when two applications are combined. It can be seen the two stand-alone applications share many components. This is not a coincidence. The smaller and more general purpose the building blocks, the more likely they can be re-used in different contexts, which leads to a more granular compatibility both within and between applications. This also explains why the original multi-purpose Lego blocks are much more versatile than today’s highly-specific ones!

app merged

We’ll be seeing this image again in a later post where we talk about compression and latency - two key issues for web-hosted software (SaaS). There are many other reasons for being happy to see so many shared components in the new combined application. Setting aside the footprint issues mentioned above, here are some of them:

  • Graphical consistency
    The combined application shouldn’t look like a patchwork quilt
  • Behavioral consistency
    Clicking on the same button in different contexts has the same effect
  • Customization consistency
    The developer (and in theory the user) can customize components.
  • Data consistency
    It should be possible to share data between the two sub-applications
  • Easier development
    Using existing components represents less work than creating new ones
  • Less testing
    Existing components are more likely to be stable

The fourth point, data consistency, has major ramifications. One of the key differences between desktop and web applications is their relative levels of interconnectivity. Desktop applications are more like islands: information can travel between them, but the journey is often not easy. Web applications, on the other hand, frequently have data integration at their core - this a direct consequence of the web’s open and multi-point architecture. Much has been made of ‘mashups’, where data from diverse sources is presented on the same web page, but this still represents a mainly read-only situation. In the future, it will be possible not only to read, but also share, organize and integrate information from disparate sources, all from within the same application.

Application Ecosystems
Just about every web site connects to a some sort of database. Most of the time, the only way to access the database is via the website, although some services now make a distinction between the two (Twitter is a good example). The micro-component architecture allows data sources to be viewed as components themselves (each with its own access permissions). And since LM applications share so much low-level DNA, it will be possible to seamlessly integrate information from different sources within the same application. This means going beyond simply displaying the information on the screen, and transforms applications into something closer to mini-platforms - with the difference that they do not necessarily have to remain isolated. To return to a geographical analogy: an application can start off as a small island, which is then gradually integrated with other applications to form ever larger and more complex land masses: bigger islands, continents, and - why not? - even planets!

This potential for combining and re-combining functionality and data sources into ever richer (or more focused) applications is exciting. There are some pretty clear evolutionary analogies, which open-up the possibility of dense application ecosystems emerging. But let’s not get ahead of ourselves. To flesh-out these concepts, we’ll present two examples; one generic and one specific.

In the image below we can see three stand alone applications. For simplicity we have each application referencing only one data source, but there is no reason why they should not reference more. We then create a new application by combining the first two. The merged application now references two distinct data sources and effectively creates a (bi-directional) bridge between them. In the final example we combine the third stand alone application - effectively super-imposing the three sets of functionality and data access within one unified application.

DS 1

Abstract examples are fine, but it’s quite hard to grasp what’s actually going on. So, in the image below we render the three applications more meaningful. We start off with a contact manager. It doesn’t do a great deal apart from allowing us to effectively organize and annotate (generic) contact information. Perhaps it includes modules (components) for communicating with our contacts: email, IM, etc. The second application is a customer relationship management (CRM) system. It’s a lot more specific than the contact manager, but is focused almost entirely on logging and measuring customer engagement. It’s not particularly good at organizing contacts or communicating with them, so it makes a lot of sense to merge the two applications. The operation is not necessarily painless, but owing to the fact that both applications are built from the same basic blocks, a lot of low-level compatibility issues are either eliminated or reduced. Finally, it also makes sense to plug-in the billing system. There’s not much point having a separate system which relies on the same underlying data (contacts). Hopefully, you’ll agree that the resulting application is more useful than the sum of its parts, and, assuming the three independent applications already existed, it will have taken less time to build than a brand-new application. Not many wheels were re-invented.

DS 2

Perhaps this last example throws up more questions than it actually answers: how is the data merged? Is it actually merged, or is replicated across the three databases? If it’s the latter, does this not give rise to normalization and integrity issues? Can the three stand-alone applications continue to access their original data, or is the combined application incompatible with its ancestors’ data sources? All good questions. Our reply is that the architecture merely facilitates the integration, it doesn’t do it for you. It provides entities such as generic hierarchical data structures (which can contain any arbitrary field format- including references to other data structures). It provides techniques for combining and extending data definitions (which are separate from the data structures themselves). In other words, it provides very general tools (which will be described in detail in later posts) to facilitate this type of integration - but the developer still takes the key design decisions, and ultimately dictates the functionality and information architecture of the new application.

LM-entary, My Dear Watson
Theoretical physics has shown how a small number of fundamental particles can give rise to the hundred or so atoms in the Periodic Table, which in turn give rise to millions of different molecules. It’s strange to think that everything that surrounds us is essentially a specific recombination of the same very small building blocks. By extension, the various interactions between matter (and even life itself) are made possible by the fact that all matter is essentially compatible at the fundamental level. This is a useful, if slightly presumptuous, way of thinking about LM’s micro-component architecture, and it gives us an idea of the diversification which we hope will be possible in the Mixable Web.

*     *     *

In the next post we aim to show this isn’t all theoretical navel-gazing - although there’s been a lot of that too - by going through a real-world (and functioning) example application.

But for the time being, thanks for reading - and if you have any comments be sure to leave them below!

Bookmark and Share Category: LM, Tech | Tags:

  • Sridharan_Srinivasan
    Your concepts and your blog information are awesome.

    I have been working / researching during my free time on something like this where applications can be built from components. My hypothesis was to test whether assembly separated/distinct from coding of an application leads to reduced complexity/increased flexibility since my focus was on reducing my pain and customer pain in executing large enterprise application developments.

    But my approach was more generic in that I don’t have any basic components, what I did was abstract the connections of the components as a language of assembly and hence built a generic framework which in theory can assemble any components using a simple generic language of assembly.

    I found that my language of assembly consists of 4 simple metadata, namely, specifying a context, adding components into the context, configuring components into the context and linking components into the context which can lead in theory to assembly oriented application development.

    I created a proof of concept web site which was fully assembled from standalone components which helped me to prove it can works, but still need to test it on complex data driven applications or even use it as you are doing to accept data from multiple streams.

    I am currently in the process of testing in a limited way whether it can used to develop a proof of concept for a large enterprise application for the architecture industry which I am currently developing for a client.

    If you have time and can check my proof of concept website www.arshu.com on my approach to solving the complexity/flexibility of web application and give me your comments, i will be greatfull.
  • Sridharan, thanks for your very kind words.

    I took a long look at your website, and I think it's very interesting. The great advantage of your approach is that a component is able to access other frameworks/libraries, for example JQuery. LM's architecture also allows this type of approach, but, to be honest, it's quite inefficient. There is a lot of overlap between the core functionality in LM's generic library and what many of these external libraries provide. Also, LM's core library is optimized for LM's own internal generic data structures and methods.

    In future posts we'll see how using generic data structures along with generic libraries to manipulate them lies at the heart of LM's architecture - it is this which permits the micro-component model to deliver highly complex applications in such an efficient manner, and allow these applications to be combined and re-combined into new applications.

    Thanks very much for stopping by, and I'll be paying attention to your site in the future.

    D.
  • Let me begin by saying I think I grok what you're trying to do and I like it. Object oriented design where all the objects share the same fundamental member structures. These base structures allow complex functionality to emerge from simple interactions.

    I've done this type of coding with reused libraries in a desktop environment. But for web programming, this architecture of resources is new to me. I have great expectations for your work Dave, and look forward to messing around with the primitives when they're ready to showcase.
  • Hi Mark,

    That's not far off, except that the components are not really objects, and the framework isn't really object orientated. All this is will be covered in later posts, but basically the system keeps code and content separate, which is not the case with methods and properties in objects.

    You've definitely grasped the key idea though: by factoring at a much lower level, there are many advantages to be had in terms of compatibility, performance and re-combination.
  • The interface are always the most challenging design elements. Localizing data, and refinement processes is subject to the particulars of the project.

    I knew you had more up your sleeve than object oriented web programming. Even in your example, we still are completely baffled by some of the forces at play on elementary particles. Their interface appears quite complex indeed, and may even show us more about the very fabric of the universe. Pardon the off topic but I read (and added my own conjecture) to some far out cosmology descriptions. Quantized space time got me curios about reality stretching, and the matter of energy
    waves/photons traversing the vacuum in comparison to gravity.

    Don't suppose you are considering a small competition to generate interest in using your language tools. There are so many fascinating free open source libs out there on my to do list, I suspect many others share my dilemma.
  • Unfortunately, LM is not yet ready to be thrown into the wild, even in a pre-alpha state. However, when it is released, we're quietly confident that it will generate some interest. Remember, LM is the only web framework with a built-in revenue model.....
  • The problem with this model is that each component have its own set of stakeholders with requirement gaps. Components tend to lean to the biggest chunks of stakeholders during its lifetime. Being a minority user of a component means recurring adaption to new interfaces enforced by majority. Backward compability maybe is not an option and createing functionality out of composite components the intercomponent glue will change constantly.
    I like the idea but the question is how practical is it really.
  • Hi Magnus, and thanks for your comment.

    I suspect you're perhaps thinking of components as being similar to an API, in other words: quite large, complex and given to change over time. Within LM, most components are very small, with limited scope for evolution. Once defined and signed-off, a given component remains fixed - it cannot change (apart perhaps from bug fixes). This means that if you build an application based on a given set of components, they will never change. There is no need for backward compatability, as they will remain in the system forever.

    If a given component is modified, then it will have a new ID (more on this in later posts). Applications which can make use of the new component can do so by replacing the old component ID with the new one, whereas developers who prefer the original component are not obliged to upgrade. This is similar to having multiple versions of the same module in a standard architecture, but is less onerous because, being so much smaller and less complex, the issues involved with using legacy components are less drastic.

    Since the architecture is based on open source, should a component evolve in a direction which is not useful to a developer, then the developer is free to fork that component. If the component were large and complex, this would be a major undertaking - but, as mentioned, this is not the case. LM components are so small, that forking them will be very straighforward.

    As (hopefully) will be clear in later posts, just making the components smaller does not completely insure against the issues you raised. The entire internal architecture of LM is designed to be generic, which means that it should never have to change. You can think of it as a completely general core surrounded by specific components which can access it. I appreciate some of these ideas may seem a bit fanciful, so perhaps it's worth pointing out that we do have a working prototype! Making the core as generic as possible is the single greatest challenge we face at the moment.

    Thanks for giving me the opportunity to talk about these (important) issues, and I hope things will get clearer with the future posts.

    D.
  • Hi David,
    I wish I had read more before I made the my comment. As you mentioned I was thinking of larger components. Now when I read further it seems to be a PHP framework.
    I have allways liked generic code in it's right places. When designing systems I try to see customer requirements as special cases of a generic solution. When abstracted enough perhaps several use cases can be solved with one generic design. I know this isn't by the book with regard of some of the Agile thoughts but it works.
    I think it is extremenly important to analyse/split what is custom and what is configuration of generic/standard otherwise customisation will spread through the system when it should have been configuration.
    I will follow this closely with great interest.
  • For the moment, the server-side code is written in PHP, but that will probably change in the release version. I realize that it's practically impossible to fully grasp how the architecture works given the very high-level description in this post. The subsequent posts should gradually rectify this.

    Again, you hit on a key issue when you talk about the tension between the generic nature of a component and the ability to customize it for particular use cases. Future posts will address this issue, but for the time being all I can say is the architecture has several ways of 'abstracting away' the differences between specific (customized) instances of the same component, and then reconstituting in the browser the specific component instance from the generic definition (I hope this makes some sense).
blog comments powered by Disqus

Back to top