Yes, this clearly calls for coffee. Or a beer.

Regarding OpenSearch, I actually think that’s an example of hypermedia interaction, not RPC. You’re GETting the Opensearch document or keeping a cached representation around. I think is is *very* different from compiling stubs & skeletons. But elaborating beyond this is probably for another time.

On the RPC stuff all you say is correct as far as the underlying technology. But I tend to look at this along the lines of the wave/particle duality of light (sorry for sounding pretentious). Use the viewpoint that works well for what you care about (e.g. light as waves for studying interference). I think there are many practical purposes where Google usage is more akin to RPC.

Look at the opensearch box in your browser. It doesn’t re-discover the contract every time. It has loaded an interface description (in the opensearch format) which is specifically defined as a “search” interface. Same thing for all the sites that have a “search this site via Google” feature. They don’t do a GET on google.com, they have a hard-coded mechanism to build a Google search URL that restricts the search to the site at hand. It is not very different from “a Search interface that was pre-compiled into stubs & skeletons”.

So I think the RPC and the hypermedia models apply best to different ways to use Google.

In the context of CMDB and config stores (what I really care about here, the Google parenthesis is more of a distraction), it’s hard to imagine that most clients will take a hypertext approach (let’s go to assetmgmt.mycompany.com and see what page I get back) versus a contract (implicit or explicit) drive approach to integration. At least for the next few years, they’ll be in effect an implicit “search/query” function for system to system integration. Maybe not for the ultimate human interface. But I am mostly focused on automation so I care more about system to system than UI in this context.

On the rise of “network protocols” (or “on the wire protocols” as I usually refer to them) you’re spot on and it’s a good thing. But POP and IMAP (and FTP) are also on-the-wire and they are closer in feel to RPC than to hypermedia. Though I agree that hypermedia systems in general make for better on-the-wire protocols because of fewer shared assumptions. And that indeed HTTP+URI is a culmination, at least for now. But it’s still wondering what exactly of this mechanism provides what value and how applicable it is to different applications. Which is what this blog series is all about.

I don’t disagree with you on WSDL, though it is especially WSDL 1.1. Technically WSDL 2.0 doesn’t force this. But in practice WSDL is WSDL 1.1 (even if people use 2.0, they use it in the 1.1 mental model). The decision in WSDL 1.1 to assign a single URI to a port (and therefore prevent fine-grained URI-based addressing) is what let to the need for EPRs and potentially the single most damaging decision in the SOAP/WS stack. Sad, really.

On the change/abstraction question, I share your hope. But I must say that I am a bit dismayed on the current focus on hypervisor-style, VM-based virtualization (“fake machines” as opposed to true “virtualized machines”). I understand all the practical benefits, but I am also pretty sure that the VMware-style VM is not the right building block for this abstraction. It will take a while to get to the right level of maturity. Middleware might be about to become cool again…

Thanks again. Your comments are helping shape up the wrap-up of this series (part 3).

- WRT RPC, I still think there’s a major disconnect. I really wasn’t attempting to focus on an OO here. I was saying, fundamentally, the Web is not like RPC in *at all* except that they share Request-Response semantics, and that you could layer an RPC interface on top of the Web if you’d like, but the Web itself isn’t one. I’ll take a step back and provide broader context, some of which I’m sure you’re aware of, forgive me if it comes across as pedantic…

Firstly, with Google, there’s no interface defined as “Search”. The interface is uniform – GET, POST, etc. To know what method to use and what data to send, the client has to dig into a different interface layer (i.e. into the data representation) and look at what hyperlinks exist there, and in what context. All interactions basically start with either a GET to a bookmarked URI or with a prior cached representation, and flow from there.

That’s very different from a model where you have a Search interface that was pre-compiled into stubs & skeletons. Which is why I was suggesting the Web is much more like DII or COM Automation or Reflection, wherein you don’t pre-compile against a concrete interface, you query the interface, match strings, match argument lists, etc. It’s much more dynamic (and, in practice, feels cumbersome, because it was usually an afterthought to the original RPC model).

Secondly, by “object-model”, I really meant “data-structure & memory-model”. RPC largely was about marshaling programmatic data-structures with portable serialization formats like XDR. The primacy of the interface, however, was oriented around the shape the server programmer wanted, with the primary intent being to “hide” the network interaction and make it look like a regular programmatic interaction. In SOA terms, it was a “producer-oriented contract”. Clients were forced to conform to what the server wanted; the burden was on them.

Whereas, network transfer protocols, like FTP, HTTP, NNTP, etc., weren’t designed by programmers so much as network engineers. They are “Network APIs”, in that they weren’t based around any particular programmatic data structures or serialization formats. They used text and/or MIME entities, and were designed around the intent of the client (a “consumer-oriented contract”). The intent was to provide the network interaction as an explicit abstraction. This arguably leads to greater scale and reliability for client developers, at the cost of increased programmatic inconvenience for server developers who couldn’t dictate things to the client. HTTP+URI arguably is the culmination to date of these many decades of work on transfer protocols (with URI being the primary innovation).

- Regarding Forms. These can be (and are) machine readable, though most of the ones we’ve seen in practice have been intended for hybrid human/machine consumption (XForms is a good example, but so are XHTML forms). Forms are the equivalent of describing an interface the way CORBA DII, COM IDispatch or java.lang.reflect.* do. Even an AtomPub interface describes a this kind of interface at a coarse grained level (“put image/* things here that belong in such-and-such category”).

What’s missing is a widely standardized way to denote, in a machine readable way, what the intent of that form is. There are practical approaches out there: RDFa, Microformats like XMDP, or eschewing XHTML and just using well-specified JSON or XML hypertext documents.

As to whether you describe an interface in MIME types or WSDL, there is a big difference. Describing a document with a schema-language, like “XSD”, is one thing. Even using SOAP-sans-WSDL would more be reasonable (using SOAPAction to denote intent). Both of these arguably can preserve the interaction model of hypermedia. But, WSDL is about describing a different interaction model that just happens to tunnel on top of HTTP. One describes operations and ports to note a variety of message exchange patterns. You’re jumping out of hypertext land and into Operation and Ports land all of a sudden. Is it bad? Perhaps not, if that’s what is needed. I often joke that WSDL needs REST anyway, since RESTful use of HTTP is the most common way to discover WSDL documents (i.e. surfing to your SOAP gateway with a ?WSDL query string). But it’s quite a jump in both infrastructure and processing model.

- You suggested:
“Agree that relationships can be made resources. It’s just that this is not predominant in REST systems.”

The nascent systems you’ve analyzed in the Cloud API space, I agree. But, I suggest taking a look at how social syndication & aggregation sites like Facebook, Friendfeed, etc., provide lifecycle and metadata services to information relationships. Those things are absolutely using machine-readable data to do it too (even if the bulk of the content is being presented to an end-user).

- WRT changes .. I figure we’ll come up with a good abstraction model as an industry some day. It might not be hypermedia-driven, but I hope it’s as extensible. I have no idea when this will happen. I do hope the work we’re doing will contribute to progressing this in some way, but after learning many hard lessons …. there’s a long, bumpy road ahead.

Thanks for the detailed comment. Here are a few responses to some of your many points:

- WRT to RPC I don’t use that term in a way that implies OO. I just mean that you call a procedure by passing its input parameters and you get results back. Basically the way good old Sun RPC was defined. With no assumption of long-lived objects or any other persistence on the remote machine. In the Google case, I call a procedure called “search” by passing as input a set of keywords. I get back a list of URLs with a small description for each. It seems to match pretty well by that definition of RPC.

- the “forms” part is interesting, but how does this translate to machine-to-machine interactions? How is my CMDB, for example, going to realize that this page represent an event notification subscription form and that this other one represents a query on the historical config of the resource? Unless you turn these forms (exposed via a common interface) into machine-readable interfaces of their own? Whether you do it though mime types or WSDL doesn’t matter as much as the fact that you have to have an agreement if there is no human sitting in front of the machine. What am I missing?

- WRT to the pre-1995 web, I agree that technically we have the same web. But in practice we don’t because of the work Google and Microsoft have put into providing these search services. Sure they are just gigantic web apps, but in practice they are part of the Web and they shape (in many ways) how web apps are created and how they are consumed. And what the web is useful for.

- And I very much agree that “there’s something very powerful about being able to access and analyze information without requiring portions of the “middleware” to understand much of its underlying schema or semantics”. For one thing, that’s what made Google possible. But not just that.

- Agree that relationships can be made resources. It’s just that this is not predominant in REST systems. But nothing stops REST-for-IT from doing it.

- Yep, there are hacks to try to deal with stale references, like the semweb approaches you list. BTW, WSRF had a child spec called WS-ReferenceRenewal that did just that but was never published. GGF wrote WS-Naming for that too.

- Finally, wrt to enacting changes, I have to some extent covered this in part 1, when looking at the IaaS cloud APIs. One of the issues is the difference between the config you can get and what you can set. I don’t think it’s impossible but it takes a level of abstraction and templatization that goes beyond the current “make it look like a physical machine” approach. But some smart people may well figure out the right abstraction model, on which such a hypermedia-driven control system can be effective. Do you happen to know any such person?

Long response. I’ll comment on this post bottom-up, if you don’t mind.

“But the query operation itself looks fundamentally RPCish to me, just like my interaction with the Google search page is really an RPC call that happens to return a Web page full of hyperlinks.”

This is a fundamental disconnect. The Google search page is not an RPC call, at least not in the usual sense of statically binding to a pre-agreed contract that is oriented around a programming platform’s object model and call stack.

One has to GET the hypertext document that describes how to generate a search URI or how to generate a search media type that’s POSTed to some kind of processing research. The closest analogy to RPC-land would be a CORBA DII, COM Automation, or Reflective RMI call. In SOAP-land, it would be the equivalent of loading the WSDL at runtime, parsing it, figuring out how to call the appropriate operation, and then invoking that operation. Yes, it sounds onerous, but I find that’s usually because you’re trying to fit an RPC mindset onto something that is quite different. Things like DII and Automation were annoying to work with because they were bags on the side of static RPC systems, and not fundamental to their respective architectures.

In your exploration of hypermedia, you’ve been neglecting the importance of its ability to describe interfaces, like forms, dynamically. When you ask, “Want to take bets about when a Cloud API URI format with an embedded regex first arrives?”, this basically underscores the problem with “Cloud API’s”. What web application has ever succeeded by standardizing URI format?

One of the benefits of a Google-like search interface is that it’s not a query interface, it’s a uniform interface that’s exposing a search gateway. Architecturally, these place major responsibility differences on the client. One requires deep a priori knowledge of the schema; the other arguably doesn’t.

“[A query] can return results that open a world of RESTful requests to you, but the query invocation itself is not RESTful. And that’s OK.”

BTW, I agree with this in principle, that Query services (which I believe aren’t overly RPC-like, but that’s another matter , could generate hypermedia docs, even if the query invocation itself isn’t RESTful. That’s OK, and a great example of this in the SemWeb world would be a SPARQL endpoint, something that really does tunnel a query interface through HTTP.

Earlier in your post, you say that REST doesn’t help solve:
“-1- The ability to query: “show me all the WebLogic instances that run on a Windows host and don’t have patch xyz applied”. You don’t have much of a CMDB if you can’t answer this. For an analogy, remember (or imagine) a pre-1995 Web with no search engine, where you can only navigate by starting from your browser home page and clicking through static links step by step, or through bookmarks.”

That’s the same web we have today, the difference is that Google and Microsoft have built multi-billion dollar businesses automating this spidering for us – caching, analyzing, and indexing the results along the way.

There’s something very powerful about being able to access and analyze information without requiring portions of the “middleware” to understand much of its underlying schema or semantics. That is what enables longevity in software, enabling the segregation of interface definitions, so that usage and formats and semantics to evolve on one plane, while another plane remains stable. That’s arguably the power of a lot of the container-oriented hypermedia formats out there like Atom, Atompub, etc. – they give us the ability to grow incremental infrastructure that will be useful for *decades*, such as the ability to manipulate and present collections of content, without having to reinvent the wheel for every sub-sub-IT-industry segment.

“In hypermedia systems, the links are usually part of the resource representation, not resources of their own. In IT management, relationships/associations can have their own lifecycle and configuration properties.”

I’m don’t see how this is any more tricky in a hypermedia system than in any system that manages data. We’ve all seen E-R data designs that don’t enable lifecycle and metadata associated with their relationships when they probably needed it. What resources are made available & how their are linked is up to the underlying servers that support those resources. There are plenty of web content management systems where a link has rich life-cycle and configuration information associated with it.

“Be careful that you can really maintain the address of a resource. It’s one thing to make sure that a UUID gets maintained as a resource configuration changes, it’s another to ensure that a dereferenceable URI remains unchanged. For example, the admin server of a cluster may move over time from one node to another.”

Managing long-lived identifiers has long been a problem in data management systems, and certainly RESTful solutions don’t solve this, because arguably it’s inherently a social / governance problem, i.e. maintaining persistent/quality identifiers is fundamentally something that requires economic investment, not technological wizardry.

To this specific case though, a variety of approaches have been suggested, from the “Cool URI’s” approach for the SemWeb, to PURLs to non-dereferencable URIs that are related via owl:sameAs. One could also opt for the DNS approach that GSLB takes – ensure at least one host remains stable in the face of an IP change, and pray that caching works out.

One final note:

Your focus has been very much on “querying”, i.e. classic CMDB chores. But what about the “management” part of configuration management, i.e. actually enacting change on these resources? Hypermedia advocates exposing a variety of links for such state-transitions, along with potentially unique media types to describe interfaces to those transitions. In other words, it would potentially be a way to enable our configuration management systems to actually DO things with their configuration data, in a uniform way, not just analyze it.

This basically what I’ve found lacking in the WS-RM or WS-Man approaches, something I noted in a longish blog post almost a year back that I distilled into four general guidelines.

Basically it comes down to, let’s really look at hypermedia as a new general model for building distributed systems, something that tends to have attributes over distributed objects or XML document exchange.