AJAX+REST as the latest architectural mirage

If the Web wasn’t tragically amnesic, I could show you 15-year old articles explaining how XSLT was about to revolutionize Web applications. In that vision, your Web server would return an XML file with all the data; and alongside that XML, an XSLT (which describes how to transform the XML into HTML). The browser would run it on the XML data and display the resulting HTML. Voila! This was going to bring all kinds of benefits over the old server-spits-out-HTML model. The XML could be easily consumed by other applications (not just humans) and different XSLTs could be used to adapt to the various client platforms.

This hasn’t panned out. At least not in that form. Enters AJAX. The XML doc is still there, though it usually wants to be called JSON. The XSLT is now a big pile of JavaScript. That model has many advantages over the XSLT model, the first one being that you don’t have to use XSLT (and I’m talking as someone who actually enjoys XPath). It’s a lot more flexible, you can do small updates and partial page refresh, etc. But does it also maintain the architectural cleanliness of having a data API separated from the rendering logic?

Sometimes. Lori MacVittie describes that model. That’s how the cool kids do it and they make sure to repeat in every sentence that their Web app uses the same API as 3rd party apps. The Twitter web app, for example, is in this category, as Mike Loukides describes. As is Apache Orion (the diagram below comes from the Orion architecture)

That’s one model, and it is conceptually very elegant. One API, many consumers. The Web site of the service provider is just another consumer. Easy versioning. An application management dream (one API to manage, a well-defined set of operations and flows to test, trace and diagnose). From a security perspective, it offers the smallest possible attack surface. Easy interoperability between different applications consuming the same API. All goodies.

And not just theoretical goodies, there are situations where it is the right model.

And yet I am still dubious that it’s going to be the dominant model. Clients of the same service support different interaction models and it’s hard for a single API to work well for all without sprawling out of control (to the point where calling it “one API” becomes a fig leaf). But if you want to keep the API surface small, you might end up with chatty apps. Not to mention the temptation for service providers to give their software special access over those of their partners/competitors (e.g. other Twitter clients).

Take Google+. As of this writing, the web site is up and obviously very AJAX-driven. And yet the API is not available. There maybe non-technical reasons for it, but if the Google+ web site was just another consumer of the API then wouldn’t, by definition, the API already be up?

The decision of whether to expose the interface consumed by your AJAX app as an open API also has ramifications in the implementation strategy. Such an approach pretty much rules out using frameworks that integrate server-side and browser-side development and pushes you towards writing them separately (and thus controlling all the details of how they interact so that you can make sure it happens in a way that’s consumable by 3rd parties). Though the reverse is not true. You may decide that you don’t want that API exposed to 3rd parties and yet still manually define it and keep your server-side and browser-side code at arms length.

If you decide to go the “one REST API for all” route and forgo frameworks that integrate browser code and server code, how much are you leaving on the table? After all, preeminent developers love to sneer at such frameworks. But that’s a short-sighted view.

Some tennis players think of their racket as one tool. Others, who own a stringing machine, think of the frame and the string as two tools, that they expertly combine. Similarly, not all Web developers want to think of their client framework and their server framework as two tools. Using them as one, pre-assembled, tool may not provide the most optimal code, but may still be the optimal use of your development resources.

There’s a bit of Ricardian angle to this. Even if you can produce better JavaScript (by “better” I mean better suited to your need) than the framework, you have a higher Comparative Advantage in developing business logic than JavaScript so you should focus your efforts there and “import” the JavaScript from the framework (which is utterly incompetent in creating business logic).

Just like, in Ricardo’s famous example, Portugal is better off importing its cloth from England (and focusing on producing wine) even though it is, in absolute term, more able to produce cloth than England is.

Which contradicts Matt Raible’s statement that “the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn’t be developing your UI).” His characterization is sometimes correct, but not absolute as he asserts.

I wouldn’t write Google+ with ADF, but it provides benefits to large class of applications, especially internal applications. Where you’re willing to give away some design control for the benefit of faster development and better-tested JavaScript.

Then there is the orthogonal question of whether AJAX technologies are well-suited to a RESTful architecture. You may think it’s obvious, since both are natively designed for the Web. But a wine glass and a steering wheel are both natively designed for the human hand; that doesn’t make them a good pair. Here’s one way to plant doubt in your mind: if AJAX was a natural fit for REST, would we need the atrocity known as the hashbang? Would AJAX applications need to be made crawlable? Reuven Cohen asserts that “AJAX is quite possibly the worst way to consume a RESTful API”, but unfortunately he doesn’t develop the demonstration. Maybe a topic for a future post.

“Because that’s the way it’s done now” was a bad reason to transform perfectly-functional XML-RPC into “message-oriented” SOAP. It also is a bad reason for assuming that your Web application needs to be AJAX-on-REST.

I’ll leave the last word to Stefan Tilkov: “Don’t confuse integration architecture with application architecture.” His talk doesn’t focus on how to build Web UIs, but the main lesson applies. Here’s the video and here are the slides (warning: Flash and PDF, respectively, which is sadly ironic for such a good presentation about Web technology).

10 Comments

Filed under API, Application Mgmt, Everything, JavaScript, Middleware, Mobile, Protocols, REST, Tech, Web services

10 Responses to AJAX+REST as the latest architectural mirage

  1. This one API and many clients always results in terrible non-contextual (beyond the data model) UI’s – the ones that are found in many system management tools that use some common management data collection technology such as JMX. Once development heads down this approach that tendency is to make the model generic enough for usage and some what self describing typically consisting of two concepts: entity + attribute. This only really works when meta data can be attached (or annotated) and the clients actually understand the meta data more so the model is attempts to describe. I understand the simplicity to development but to the user it generally results in a horrendous interface and interaction style unless you have a very experienced UI designer that avoids this trap (which sadly is rarely the case).

    Anyway I think we are going to see the boundaries between what is deemed client and server domain diminish as both client and server runtimes become much more self aware and self regulated moving computation around different execution container units blurring what is cloud computing and what is public or private.

  2. Interesting blog post. All I can tell you after 4 years of active AJAX+REST development both on desktops, phone and tablets is that it just works and it is a lot cleaner, efficient and flexible that XML, SOAP and XSLT will ever be. Does it mean that it is the holly grail and you can not create spagetti apps…absolutely not…You still need to design solid data models, find the right granularity, sometimes offer multiple facades to multiple clients, optimize for batching and larger data set, manage exceptions, have a solid caching strategy, etc.. etc… but all these things feel a lot simpler if you understand and use JSON+REST properly…once you have the contract in place, implementing Javascript clients is simpler because JSON+REST are a lot more lightweight in the browser.

  3. Interesting thing that you happened to mention XSLT, because there actually exists a highly AJAXified web framework using XSLT as templating language.

    It is called Context and XSLT has been the key factor of making it flexible and especially suitable for partial page updates. It also integrates with Javascript nicely. You can check more from http://www.contextfw.net.

    • First of all, thanks for the article, really well done.
      I was about to write some remarks regarding how the separation from data and visualisation was first taught and then pursued by me, and how difficult is to gain such separation on AJAX technologies.

      Then I read this reply about Context that I must now try to check and see for myself :)
      Thanks you too Marko.

  4. Stu

    Curious. XSLT was very much about declarative transformation, and the benefits of expressivity and brevity. Whereas JavaScript is often just imperative templating. If anything, JavaScript and XHR seem to be the resurgence of client/server programming, except the data is now (possibly) self descriptive and (sometimes) distributed. I look at XQuery and DataTables, and I think we have the power and productivity of the PowerBuilder Data Window without the maintenance and scalability headache.

    Secondly, as l long as you retain some sort of semantic integrity in the data exchanged, I think building different interaction styles into different sets of resources is one of the great strengths of a resource-space approach to APIs. Whether this is “sprawling out of control” is a matter of the implementation. So long as the interface is useful to people, and the data remains interoperable across different interaction styles, why does it need to be organized in any particular way?

    Having built at least one product this way, and just shipping a new major order management system for my current enterprise this way, I don’t want to go back to unified models or web page generation. A set of static pages and JAX-RS services enable tremendous productivity gains and often result in a very clean service interface for interoperability. Basically, I see this becoming the dominant model on the Internet and with unified framework approaches being the niche for businesses and intranets that are just building apps, not trying to build interoperable data platforms.

    I do think of course there’s room for improvement, particularly raising the bar beyond CRUD apps into describing the impact of POST on data when it does more than create a record, but I’ve been saying that for a while, and am still working on an approach that I want to actually deploy on something useful before spouting about it.

  5. Unified APIs like twitter is possible, in very simple cases at least.

    Let’s go back and see what twitter did, and what does it mean from an architectural standpoint.

    First off, twitter’s REST API is not REST. You can call it like that, but it’s what I call “URL-RPC”, where you basically have a host/service/method URL, and you pass parameters to it. Session handling is kept to the bare minimum.

    Now once you did the favor for yourself of defining everything as a set of service classes, which always return a snapshot object, you can basically bring it to anyone.

    For web developers, this is incredibly familiar. If you do it with Ruby on Rails, or pretty much any PHP system, you have done this since the early 2000s, it’s only it’s that you don’t provide the templates. You let the templates run on a different architecture.

    Step 1: So, application architecture-wise, you do a full web application the way you always did it,it’s only that you introduce a separation which will be so automatic that it’ll be automatic in your code at the first level. Instead of your services returning HTML as they always did, the service returns with all the data you’d put in the template, it’s only it’s sans the template.

    (Once you introduce XML, you’ll have an XML template, which is not good for you)

    Any code monkey with a month of lurking on forums can understand this level and is able to successfully create an architecture which does that. That’s why it’s so widespread. If everyone can understand it, it’s pretty easy to re-implement about a million times.

    Step 2: AJAXification (with the HIJAX pattern) usually also worked the same way. Normally, you create a lot of little modules, and on each load, you call them one-by-one. With AJAX, the first thing what has changed, that you created a service, which, instead of returning the root template, then filling each of its childnodes with the module service’s template, you returned with only one module. Simple, easy. Then you somehow eliminated the module-service distinction, and each module was able to be called separately from the outside. Boom. HIJAX done. Takes about two weeks to implement for a script kiddie. You can do that too, therefore.

    Combine the two, and basically you have a full-fledged API.

    So, that’s how twitter API was born: Simply what was done anyway went back to another layer, and automatically exposed.

    Step 3: The next step was realizing, that anything you do, there are two computers involved: the server and the client. Publishing the templates to a different directory was a straightforward manner: once you did step 1, you only had to convert your thin layer between services and templates into javascript (and by then, as you pushed these a level back in Step 2, you already had a thin layer of combining these services, you had to port this one too), which, by the time doing this, was a relatively easy thing to do.

    TADA! You have the universal API. You’re done.

    What needs to be understood, that this is not an enterprise setting: twitter people didn’t want to support various services, they said, well, this is what we use in our application, and we make no barrier for you to use these too, but don’t complain on the free food, we won’t develop services just for your amusement. Every single service was used by the main twitter app itself.

    Also, these web frameworks are pretty straightforward, because they were to solve web people’s needs who understood how web works: compared to some java frameworks, which came from a different direction, and wanted to port the usual things to the web (eg. visual UI designer tools). Therefore, these couldn’t have born in the java world, as java was about avoiding/hiding the problems these frameworks were to solve.

    They didn’t mix their application infrastructure with their network infrastructure. The realization, that a special network infrastructure is built into the situation, and they can make an advantage of it is the last step, everything before that was just a kind of natural evolution!

    Whatever you do, if you’re doing a web application, you have a web client and a web server. That’s the definition of how the web works (OK, actually, the definition is REST [Fielding, 2001], but part of REST is being client-server based).

    If you try to hide this, you can, originally, they hided it too. But once you don’t hide this, but make an advantage out of it, and you throw out everything which wants to hide this, you can build such architectures. Because this single constraint is built into the system, it’s not an optional architectural part.

    And a last word: I have nothing wrong with XSLT, I love that language. SOAP and WSDL have some ingenious ideas, which I really miss being standardized right now. Perhaps the reasons for the missing standardization is that with the way how non-enterprise web frameworks work, it’s a no-brainer to do these 3 steps, it’s kind of a boilerplate. I’m not claiming it’s right, I was writing my thesis about the URL-RPC API standardization based on all the goodies WSDL/SOAP have brought us, yet this is what we have now.

  6. Chad Wackerman

    You had me until you referenced XML-RPC which was clearly created by someone who understood neither XML nor RPC.

    JSON got rid of the schema, and REST got rid of the verbs. And after a billion dollars in funding, Twitter.com still hangs and displays partial pages. Perhaps “loosely connected, typeless, schemaless” isn’t the way to go after all?”

    • Jason

      Perhaps one websites usage of a technology is not the defining example you wish it to be? Twitter is a train wreck, it is also a horrid example to try and bash JSON/REST with. Chad you very clearly do not know what you are talking about

      • Chad Wackerman

        The only technology I bashed was XML-RPC. If you’d like to defend that idiotic lump of coal, I’m all ears.

        Obviously nothing about REST demands that you use typeless languages to post schemaless data. Nor does anything in REST require companies to wrap their “API” in a half-dozen language-specific SDKs, but they all seem to do that, too.

        The tools and frameworks are primitive and people are a little too focused on the browser and consumption patterns, not architecture. The XML-RPC quirk of this article aside, that’s what I took from it and agree.

  7. Anon

    It also kills agility. At my last job this was the hospital as handed down from the architect – “we will have to write the api anyway, why not do it first and run the site off it?”.

    The whole point of an api is that it doesn’t change – if out does third party developers get very angry. This means that any features you roll out and are unpopular, any properties you rename still have to be supported.

    Some slick code for versioning the api, for grandfathering features and good client error handling can help, but before you tell 1000 developers about a new api call be very sure that that is how you want that feature to work.