Square peg, REST hole

For all its goodness, REST sometimes feels like trying to fit a square peg in the proverbial round hole. Some interaction patterns just don’t lend themselves well to the REST approach. Here are a few examples, taken from the field of IT/Cloud management.

Long-lived operations. You can’t just hang on for a synchronous response. Tim Bray best described the situation, which he called Slow REST. Do you create an “action in progress” resource?

Query: how do you query for “all the instances of app foo deployed in a container that has patch 1234 installed” in a to-each-resource-its-own-URL world? I’ve seen proposals that create a “query” resource and build it up incrementally by POSTing constraints to it. Very RESTful. Very impractical too.

Events: the process of creating and managing subscriptions maps well to the resource-oriented RESTful approach. It’s when you consider event delivery mechanisms that things get nasty. You quickly end up worrying a lot more about firewalls and the cost of keeping HTTP connections open than about RESTful purity.

Enumeration: what if your resource state is a very long document and you’d rather retrieve it in increments? A basic GET is not going to cut it. You either have to improve on GET or, once again, create a specifically crafted resource (an enumeration context) to serve as a crutch for your protocol.

Filtering: take that same resource with a very long representation. Say you just want a small piece of it (e.g. one XML element). How do you retrieve just that piece?

Collections: it’s hard to manage many resources as one when they each have their own control endpoint. It’s especially infuriating when the URLs look like http://myCloud.com/resources/XXX where XXX, the only variable part, is a resource Id and you know – you just know – that there is one application processing all your messages and yet you can’t send it a unique message and tell it to apply the same request to a list of resources.

The afterlife: how do you retrieve data about a resource once it’s gone? Which is what a DELETE does to it. Except just because it’s been removed operationally doesn’t mean you have no interest in retrieving data about it.

I am not saying that these patterns cannot be supported in a RESTful way. In fact, the problem is that they can. A crafty engineer can come up with carefully-defined resources that would support all such usages. But at the cost of polluting the resource model with artifacts that have little to do with the business at hand and a lot more with the limitations of the access mechanism.

Now if we move from trying to do things in “the REST way” to doing them in “a way that is as simple as possible and uses HTTP smartly where appropriate” then we’re in a better situation as we don’t have to contort ourselves. It doesn’t mean that the problems above go away. Events, for example, are challenging to support even outside of any REST constraint. It just means we’re not tying one hand behind our back.

The risk of course is to loose out on many of the important benefits of REST (simplicity, robustness of links, flexibility…). Which is why it’s not a matter of using REST or not but a matter of using ideas from REST in a practical way.

With WS-*, on the other hand, we get a square peg to fit in a square hole. The problem there is that the peg is twice as wide as the hole…

37 Comments

Filed under Cloud Computing, Everything, Implementation, Modeling, Protocols, Query, REST, Utility computing

37 Responses to Square peg, REST hole

  1. I agree. A well designed REST API can be a thing to admire, but not every application fits well. Sometimes, you just need to get things built and there are often many choices. A REST-like API is my favorite style. The most important thing is to make it well though out and logical.

  2. I asked almost a year ago Is OCCI the HTTP of cloud computing?, but then I came to the realisation, like you, that in fact HTTP is the HTTP of cloud computing“.

    So what do we need to do to get you involved in occi-wg, knowing that we’re getting pretty close to our goals now so it’s mostly just a case of solving difficult problems like events?

    Sam

  3. I had a go at answering. The Web approach is easier, only because there are so many examples to see and copy from the wild, whereas WS-* is a minority sport conducted behind closed doors: http://blog.whatfettle.com/2010/02/23/square-peg-web-hole/

  4. I decidedly agree here. I get pushback whenever I suggest a non-RESTful mechanism for remote HTTP access to an application, but for some things REST just isn’t a good model. But it’s the buzzword of the day, so all the square pegs have to be shoved into this round hole even though it’s not the best tool for all jobs (for a lot of jobs, yes, but not all). I have to show huge savings in development time — as in, literally weeks of work condensed to 2 hours of work if we switch to a different model — before I can get something other than REST as the remote HTTP access. So it goes…

    -ELG

  5. Not surprisingly, I am in complete agreement.

    Please do not forget “versioning” which can’t be achieved in REST because it couples “access” and “identity”. To this date, there is no proven RESTful versioning mechanisms (you just can’t version business logic like you version a web page).

  6. WS-DeathStar is what it is, a fully-armed and operational battle station. That gets taken down by an X-Wing blast. Between polluting the REST conceptual idea to make it work, and using WS-DeathStar… ahem, wise engineering says “do the first”.

  7. I love the way your posts flush out people who think differently to my conventional world on The Web. Am now puzzling how there is not story for versioning in REST when WebDAV/svn or most any DVCS is pretty RESTful compared with the impossibility of evolving SOAP messages, especially those you’ve been fool hardy enough to describe in WSDL/XSD.

  8. Paul:

    I love your fuzziness doubled by your ignorance: “is pretty RESTful compared with the impossibility of evolving SOAP messages”, again, show me how you handle versioning of the business logic, don’t wave your hand, don’t use words like “pretty”. Just show me.

    You seem to know nothing about SOA and Web Services (stop waving your hands, again, show us the facts), so you may check the work of Dave Orchard who pioneered all these versioning mechanisms (which work). Kjell-svere and I wrote an article to summarize the approach: http://www.infoq.com/articles/contract-versioning-comp2

    Cheers,

    JJ-

  9. I don’t get what the big deal is here; without being anything like a REST maniac most of the above “issues” not only have solutions but commonly deployed ones too.

    Long-action: you return a 202.
    Query: that is inordinately common. GET /apps?name=foo;patch=1234. Note that in HTTP the query string is an identifier.
    Events: you poll, or you expose your own service that can be contacted.
    Enumeration & Filtering: you craft URIs (maybe using queries, maybe not) that point to the inside of the resource. The pointers will depend on the resource type.
    Collections: If you’re dealing with APIs that don’t allow you to operate on collections, those are broken APIs. It’s not a REST problem — REST can perfectly well support that.
    Afterlife: you just return the header (in fact, if you need to, the body) you want with a 410.

    None of that makes it good in all cases, but well, it does make for a vast majority of situations in which WS-* can be completely forgotten about.

  10. Well of course JJ is right: I am ignorant, and am enjoying hiding as much as I can from WS-* these days!

    Maybe I should offer in my defense time spent trying to make WS-* work in production software and standards fora, building test cases and test suites in particular around techniques for versioning, most latterly a report which whilst containing some flaws in execution, does at least give some indication of the practical difficulties we encountered making SOAP work: http://www.w3.org/2002/ws/databinding/edcopy/report/all.html

    As a fan and champion of DaveO’s formidable work outlining the possibilities for versioning and compatible evolution of messages, especially with WSDL 2.0 and XSD 1.1 I’d love to say issues with WSDL 1.1 and XSD 1.0 and toolkits are now long forgotten and my hand waving is irrelevant, but the experience of colleagues stuck, unable to make small non-breaking changes to WS-* legacy systems tells me otherwise.

  11. Paul:

    again, I respond to facts not hearsay, and if we were all to stick to that discipline, we would have made a big step forward. I am just tired of people saying “I heard…” or “that kinds of work…”.

    Forward versioning works in XSD 1.0/WSDL 1.1. It will work better in XSD 1.1 when it comes out…(who really cares about WSDL 2.0?).

    I have no vested interest in WS-*, I just don’t want someone to sell me something inferior to what I already have, just because they “claim” it is superior. Show me the facts and I’ll respond precisely.

    JJ-

  12. Paul:

    more specifically, with respect to the XML Schema Patterns for Databinding Interoperability Report, I don’t use any fancy or exotic XML Schema, other than the extensibility mechanisms essential to versioning. See, most people see XML Schema as a “modeling” technology, for me this is just a (poor) validation technology.

    For me, XSDs are generated from a DSL, I have given the recipe here: http://www.infoq.com/articles/message-type-architecture. I may be wrong, but IMHO, everyone that creates and manages XSDs by hand is bound to fail. I am currently working on the Parlay’s APIS and even though they are trivial you can see the semantic mashup that XSD creates with data structures that are that small. Imagine now that the data structure are a lot bigger and interelated, this can only be a disaster. It is not my fault if people don’t know how to use a technology, that’s not enough of a reason to throw away the technology.

    JJ-

  13. Umit Yalcinalp

    [Reposted from Facebook (where this blog entry appears as a note) with Umit’s permission]

    I am not as negative as you guys are. There are some aspects of your concerns already addressed. Two of our interns at SAP, one was mine, worked on several aspects you listed. One of them wrote his thesis, the other one expanded on the work. The session concept can be incorporated by making access via the session/resource coupling. Thus, the … See Moreresource is tied to the session/transaction. The relationships can be achieved via navigation encoding + query (for the variable bindings) using the existing URL techniques but retaining the RESTful manner. I believe versioning could also be achieved if you were to consider two versions of the same “thing” to be two different resources, thus using two different URLs. Ask Jan Schulz-Hofen for his thesis from Hasso Platner Institute in Germany. I can provide examples of how encoding may look like. If I am not mistaken, Microsoft Astoria, Google data api, all use very similar encoding mechanisms, although I find the former a bit esoteric.

  14. Jean-Jacques Dubray

    [Reposted from Facebook (where this blog entry appears as a note) with JJ’s permission]

    >> I believe versioning could also be achieved…

    Umit, I am sorry, but after 3-10 years of debate we need to stop the “I heard that XYZ says it works”. We just need facts. I am not sure that “consider two versions of the same “thing” to be two different resources” qualifies as a fact. Please provide the reference.

    You can always use two … See Moredifferent “bookmarks” (per Roy’s definition) as an entry point to a set of resources, but then REST is strictly equivalent to WS-*. The bookmark is just the endpoint of the service. In other words you can’t access a PO 123 directly without first navigating to the bookmark and asking for the PO 123 (where 123 is a Unique identifier, not a uniform one). Other than that, I have never seen on paper or on the Web any mechanism that actually works. IMHO, it can’t work because of the coupling identity/access introduced by the RESTafarians. Roy’s REST doesn’t have to deal with that because Web Pages do not invoke business logic to be displayed. Versioning Web page instances, is a completely different problem, that one can be resolved by URI, because in effect, they are two different resources.

  15. Umit Yalcinalp

    [Reposted from Facebook (where this blog entry appears as a note) with Umit’s permission]

    I actually did not make up what I wrote. I led the project at SAP where we were exposing data services for objects through restful interface. One of the students was my intern, and we have successfully demonstrated this with lightweight RIA clients talking to SAP backend on top of existing tx model with sessions. It does work. I wish I could give you better reference as we were in the process of presenting this last year in the Microsoft led conference on DSLs and there are internal writeups but you know what happened at that time… Not because the project did not work, it is continuing on. I can not comment on how it transpired afterwards. I will ask Jan a pointer to his thesis to send it to you. His work came before Google Data apis, etc. so they are very very similar. Matthias made it better and tackled sessions.

    Now, we did NOT tackle the data services having behavior (other than REST verbs that map to crud) such as update X when Y, K is true, but not when Z, etc. We are only talking about data services, not replicating full fledged interfaces. Actually, the methods in services could be tackled as you know with WSDL 2.0 HTTP binding. Would that be RESTful? I agree that is a big debate.

    Note that this is in the context of Data oriented services for RIA when the client encapsulates a lot of the behavior on localized data not on the server, such as checking whether amount x > y there is an error, etc. Thus, the server side behavior can be as a side effect of CRUD (like triggers) which did not need to be exposed. In these parameters and for data rich clients, the REST model was beneficial for RIA. … See More

    I am not a hammer and nail gal and this is a very practical perspective with data oriented services. Every method has pros and cons and applicability in the context they try to tackle. As a matter of fact, this is why folks got attracted to REST in the context of RIA, Json, etc. It is simply very attractive and easy to develop with for client programmers which need to deal with CRUD.

    AFA Versioning is concerned, practically two different versions via GET should be able to expose their data but older versions should no longer be allowed to update (POST). This behavior does imply that there is additional logic that v1 and V2 are indeed temporally linked and an entity (the server) has to enforce it. Is it really RESTful? I am sure David Orchard can comment for hours :).

    For any of this to work, the URL is more than an endpoint. However, each URL does signify a unique identifiable resource provided that each resource is served by the server by deciphering the URL encoding…

  16. Jean-Jacques Dubray

    [Reposted from Facebook (where this blog entry appears as a note) with JJ’s permission]

    Umit,

    I am not claiming you made it up and on a scale of 10, I would certainly trust what you claim with a 9.5. However you would agree that the question is important, quite fundamental actually and I cannot comment on a single sentence. I need a reference to be able to understand your claim.

    JJ-

  17. For concrete mechanisms for managing revisions of a resource using HTTP, in a RESTful way I suggest looking at Git (which you can explore using http://github.com) which talks to most other version control systems using HTTP in a variety of RESTful and non-REST ways, in particular the WebDAV/DeltaV protocol as used by Subversion (svn): http://www.webdav.org/deltav/protocol/rfc3253.html and patterns for managing merging versions and eventual consistency over HTTP which is explained quite nicely in the CouchDB book: http://books.couchdb.org/relax/

    As for extensibility, and compatible evolution, with SOA and WS-* I see the difficulty being with a reliance on modeling, abstraction and contracts in meta-data formats which kill evolution such as XSD 1.0 and its many historically poor implementations. So in that regard I think it’s the expectation of being able to use tools to abstract the world into your own terms, be that UML diagrams, BPEL flows or Java objects that is the cause of much difficulty.

    Compatible evolution in practice is far simpler on the Web, in particular using HTML and JSON, where there is no notion of a shared model or formal contract, rather interfaces are defined using examples. Web developers usually care about the content and form of the actual documents exchanged, and understand that adding more values is unlikely to kill existing interactions.

    Now that was hand waving, but that’s because I see this as a cultural, not just technical issue.

  18. Stu

    First, JJ has long been an anti-REST pundit, whom I retain respect for. He has many good points that are worth considering, and some hopelessly bad ones. But, as I’ve said years ago (when I stopped debating him), it’s very hard to productively debate due to his repeated stooping to personal attacks, calling people stupid, ignorant, etc.

    Second, William, when you say “polluting the resource model with artifacts that have little to do with the business at hand”, I find it curious why you would think the resource model has to be 1:1 mapped with the business at hand? In what technology has that *ever* occurred? Physical RDBMS design? SOAP APIs? The closest *might* be pure object-oriented domain models persisted via an ODBMS or a sophisticated O/R mapper. How many of those have you seen in the wild?

    On particular topics:

    Long-lived operations: Yes, an action in progress resource makes sense, as it would in a website. Or in a database with a jobs table. Or an EPR for a subscription to an event sink in WS-Eventing.

    Query: There are two styles of query here. One is where the query options are described in machine-readable hypertext and POSTed back, as how we do queries today with a Form. The other is where the query language required is indicated in hypertext, along with a description of the data model, and the agent POSTs the query language back, which is how, say SPARQL works. I’m not sure where the impracticality comes in.

    Events: This problem exists with any eventing mechanism out there, so I’m not sure what the problem is here.

    Enumeration: This is not the problem of GET. That would be like saying it’s WS-Transfer’s problem, and we don’t need WS-Enumeration. This is the problem of a media type that can handle enumeration. RFC 5005 is one solution. OpenSearch has another.

    Filtering: Design a model that allows you to filter results through a query?

    Collections: I’m curious. Why can’t you do exactly what you described? Under what RESTful tenant?

    The afterlife: DELETE expresses intent. It doesn’t mean it has to go away, (e.g. 410 Gone) afterwards. You can DELETE something, then GET it, and something might still be there for posterity. How this works is up to how the media type you’re using defines the semantics of DELETE on the particular hyperlink the agent wants to DELETE.

    This whole post has a flavour of “REST doesn’t wash my socks”, and of course it doesn’t, it’s like saying “pub/sub eventing doesn’t wash my socks”. You are holding WebArch to some kind of abstract standard that you don’t even hold WS-* to, meaning it’s supposed to solve all my problems without requiring me to engineer solutions.

    The problem with REST, and here’s where I agree with JJ, it’s that it’s not this “everything is better , solved, wonderful, and filled with puppies” world that some make it out to be. A sign that you’re dealing with one of those foks is their instance on using PUT for everything and avoiding POST, because, you know, that would require more work in defining a media type that uses POST appropriate. There’s a lot of work to be done, and a lot of standard media types that would be very helpful the machine agents that are…. missing. Atompub and its extensions are only the beginning, and it’s a bit frightening that it’s taken this long to make so little progress. But that’s mainly because people are building local solutions to their problems, can kind of sick of standards organizations.

  19. Stu:

    hi, I am neither anti-REST nor a Pundit. I am not a brand, nor wish to have “followers”, I don’t use my name as an argument. I use arguments and if I am wrong, I’ll be the first one to admit that I am wrong.

    I am not anti-REST, I am anti “everything is better , solved, wonderful, and filled with puppies world”. I fully support approaches such as the one from Subbu who simply says, if you are going to use HTTP in one way or another, here is how you should use it. I fully support Roy’s views on REST and REST as the foundation for the Web. Now when people claim that REST is “middleware” or worse, REST is a “programming model” (that comes with its own QL, asynchrony, eventing mechanisms, ***) far more superior to anything that already exist that’s where I … (fill in the blanks).

    Now when someone makes a claim such as “versioning in REST [can be achieved with] WebDAV/svn or most any DVCS is pretty RESTful compared with the impossibility of evolving SOAP messages” and that someone offers no substantiation, I feel I have the right to call him both stupid and ignorant. He/She has always the choice to do like Umit did to substantiate his or her claims.

    Our industry has been made so sick by its punditocracy (of which I am not part of, and will never be part of) that we have now the characteristics of the pharmaceutical industry (without the FDA and double-blind processes). The software industry is now in the business of inventing “remedies” to all the diseases it created by “following” without “questioning”. You get the picture, right? can you imagine what the pharmaceutical industry would look like if any one could claim whatever they wanted without “questioning”.

    So I repeat, there is no documented versioning mechanism in REST despite what the punditocrats claim laconically. Even what Umit has written doesn’t qualify as an answer, though she provides a lot of details about the context in which she made the claims. I believe her context is way too limited to define a general versioning strategy (as most of the business logic is in the RIA client, which is an anti-SOA pattern and a common problem of siloed solutions).

    The RESTafarian punditocrats could have at least the decency to revisit their claims from 2007 about “uniform interface” for instance (in the light of RESTfulie) and come forward to express how wrong they were about claim an interface to a resource is uniform. For me, until versioning is solved (and I don’t see how), for me the (other) REST (not the one of Roy) is a non starter in the Enterprise/IT. It is, so far, just a collection of recipes to use HTTP here and there to solve tactical problems. I am still waiting for the examples of enterprise solutions built in a 100% RESTful way (ERP, CRM, PLM…).

  20. Paul:

    it seems that there is a disconnect. You seem to not understand (kind of) the problem that I am referencing. I am talking about versioning the business logic behind a RESTful call (GET, POST, PUT, DELETE), not the resources themselves. I understand that versioning resources has been solved decades ago. Unless you believe that REST is directly wired to the metal, you have lots of code behind a POST or a PUT for instance. How do you version that code?

    I said “kind of” earlier because you explain “Compatible evolution in practice is far simpler on the Web, in particular using HTML and JSON, where there is no notion of a shared model or formal contract”, yet you seem to not understand what “compatible evolution” means. How can you be “compatible” without a “contract”. You just “hope” for compatibility? how can you test, certify, assert compability? without an agreed upon contract and rules for compatibility (described in the paper that I referenced earlier).

    So again, I would highly recommend you take all the “emotion” out of the question and look at the problems in their entirety while avoiding the laconic “XXX solves your problem”.

    WS-* had the merit of kind of supporting loose coupling and versioning which are both impossible to achieve in REST. We are throwing away tremendous amounts of value simply because some people don’t know how to use XML or XSD, let alone WSDL. That is sad.

    JJ-

  21. Hi Stu (I am responding to comment #18),

    WRT to “polluting the resource model with artifacts that have little to do with the business at hand” I indeed think this is something we should strive to avoid. It doesn’t mean that I am not open to compromise. Allowing some amount of abstraction leakage might be the pragmatic choice. We do this all the time, as you point out, because perfect abstraction layering (or something close to it) usually comes at a huge cost. But we have to realize that there is also a cost to allowing this pollution/contamination/leakage. So I feel justified in listing this as a drawback (not a deal-breaker).

    WRT to modeling actions in progress, I don’t have a problem with modeling a server in the process of being instantiated, or an application in the process of being deployed, as these are representations of a true aspect of the domain I am modeling. My issue is with the idea that any operation that may take some time to execute (e.g. a long-running query) fits the same model. To me, there is a difference between actions in progress that represent the realities of the domain and actions in progress that represent the technical limitations of my infrastructure. What if tomorrow I upgrade from SQL Server to Oracle DB and my queries become blazingly fast? ;-) Am I then stuck w/ model elements that are vestiges of my old infrastructure? I’d rather not have those in the model, so I only have to deal w/ model changes when the entities in the domain I manage actually change, not when my management infrastructure changes.

    For the same reason, I have no problem w/ modeling a “subscription” but I want to keep any info about how the delivery takes place out of the model. That should just be, from the model perspective, a self-described data element (e.g. a URL w/ whatever appropriate scheme).

    WRT to query, you and I have had this debate before, in the comments for this post. There, you agree with my statement that

    “[A query] can return results that open a world of RESTful requests to you, but the query invocation itself is not RESTful. And that’s OK”.

    You add that “[A query] could generate hypermedia docs, even if the query invocation itself isn’t RESTful. That’s OK, and a great example of this in the SemWeb world would be a SPARQL endpoint, something that really does tunnel a query interface through HTTP”.

    So what are we arguing about?

    I am not trying to play “gotcha”, just showing that I really value and remember the great comments deposited on this blog… :-)

    WRT to enumeration, my fear (maybe unjustified) is that it’s now the REST approach that is flirting w/ over-engineering (e.g. links for enumeration within a resource rather than for linking actual resources as in RFC 5005). Versus the simpler URL-parameter approach that Paul suggests but is less RESTful than MNot’s RFC.

    Collection: what resource do I send my “reboot instances 1, 2, 5 and 8″ message to? Do I need to model a “bag” resource that contains them all? Do I create a resource specifically for these instances and this operation? Do I keep around such large “bag” resources and pass the ids of the server to reboot as a parameter? Compare this to the simpler task of sending a “reboot instances 1, 2, 5 and 8″ to a shadowy, not-part-of-the-resource-model controller endpoint that can do things for me. In effect, for some operations I want to be able to address explicit resources in a RESTful way, but for some others (query, collection operations) I want to be able to send them “in the Cloud” to be handled by some invisible infrastructure controller. And of course there should be a tie between these two sets of invocations (e.g. the use of hypermedia in the responses). I should probably do a full blog entry on this dichotomy.

    WRT to DELETE I think this becomes again a bit complex and variable. Paul Downey, for example, seemed to think in his reply post that DELETE indeed means you only return 410 Gone on it (but you could still stuff in some info in the 410 reply). It might be easier to provide a simple way to access deleted resources than to finesse the per-media-type semantics of DELETE. Though on that one I could actually be convinced to find a way to make DELETE work as we want.

  22. Nice roll-up, William!

    One point: my understanding of DELETE from RFC2616 is the intention is to remove the resource, or make it inaccessible. I find it useful to consider an intermediary such as a proxy who on seeing a DELETE would discard any cached version of a representation of a resource, but would still honor a subsequent GET to the URI. So whilst it’s possible for the server to continue returning zombie copies of the DELETEd resource, it’s not really what DELETE means, and my suggestion is to POST/PUT a change of state to “deleted” and keep a record of the resource is usually what most people expect.

    I do find it hard to talk about “business logic” because for me that’s invariably something of an oxymoron. It’s a phrase, which along with the notion of machine readable contracts I get by these days without hearing or needing use. As for compatible evolution, three simple rules have served us all very well over the years:

    1) have sensible defaults for missing values for backwards compatibility.
    2) design formats with ignorable extra values for forwards compatibility.
    3) if you change the meaning of stuff, or the processing in a way the client doesn’t expect, then it’s a breaking change, so use a different interface.

    kthxbai!

  23. I’m with Rudd-O (#6) on this one.

    You guys better be careful — it’s starting to smell like rest-discuss in here…

  24. Umit Yalcinalp

    Here are the pointers.

    — Jan’s thesis: http://www.scribd.com/doc/2218300/WebData-Defnition-of-a-Middleware-for-Exposing-and-Accessing-Objectoriented-Domain-Models-as-Web-Resources

    — Our presentation on RIA. This does not provide the details on the protocol, but puts you into the frame of reference why/where we were using RESTful services. That is linked to my profile in linked in and probably most of you have seen it already.

    http://tinyurl.com/ycfna97

    JJ, we did not address business logic behind resource’s and their versioning. I do not claim that this approach addresses that issue or it is tied to the protocol. However, I would imagine that as business logic if it was tied to a resource were to change on the backend, it would be “logical” to expose a different version of the resource.

    Why are REST discussions are always RESTless.

  25. The real failing is the obsession that many technologist have with the silver bullet approach. REST-ful interfaces work for a great many use cases, and not so much for others (RPC, events, querying being examples discussed above). WS-*, well, let’s not go there. We’ve chosen to implement a somewhat purist REST-ful pattern for durable “things” for which such an approach makes sense, and other XML-over-HTTP, binary-over-HTTP, and JSON-over-HTTP techniques where they make more sense.

    There are similar religious debates occuring on the data storage world – NOSQL, SQL, KV stores, graph databases, columnar stores, and so on. The reality is that a well crafted architecture can blend many of these technologies. I’ve found it just as important to consider the “consumability” of your functionality as it is to consider the plumbing, with an eye on understandable semantics, self-documentation, extensibility, and support for a broad range of human/software “consumers”. Just my $0.02, after taxes.

  26. Indeed, Silver Bullet Syndrome is rampant. It’s like the language and OS wars. Should you use Linux or Windows or Solaris? Python or Perl or C or C++ or Java? In the end, a pragmatist has to say, “use what *best fits your problem set*”. Same deal with web standards. REST is useful for a large set of problems. It requires more work than it’s worth for another set of problems trivially solved by some other XML-over-HTTP or etc. protocol. There’s no reason to believe that all problem sets have a single Silver Bullet that solves them, and REST is no more a silver bullet than is Linux or Python or whatever.

  27. Stu

    William (#18),

    WRT “polluting the resource model”, I think the root of this concern is how REST (hypertext, really) combines control and content into the same plane, whereas many other architectures separate the control plane out. For example, The tradeoff when you do this, is that the content itself directs the control plane (aka. “data-oriented” rather than “process-oriented”). When we’re so used to having data in its own world and process in its own world, it can be strange to see them brought back together.

    WRT modelling actions in progress, it seems to me that evolution of long-running operations vs. short-running operations is a fairly universal tradeoff across architectures. I’m not aware of an architecture style that isolates my domain model from this sort of thing so that I can freely switch between short & long-lived with no impact to the model itself … unless I *assume* everything is long-lived and provide some mechanism to hide details for the short-lived case. This is the message-oriented middleware (MOM or JMS) way, in essence. It is built around the long-running case, even though I *could* just configure a transient, in-memory, non-persistent queue and make it look to my programmers as a regular procedure call.

    The fundamental problem is whether the action itself is a datum worth tracking for a period of time or not. If not, then it’s not in my data model, if so, then it is. The tradeoff with REST is that the data model is visible, with MOM it’s hidden in the bowels of the message server. There were plenty of similar debates over the use of MOM vs. a database for persistent queueing.

    WRT query, looking back, there are two ways to interpret the word “RESTful”. One is “approved by the REST gods”, the other is “relaxing some of the constraints, usually of the uniform interface”. I meant the latter, of course. ;-) A query interface with a query language is not really taking advantage of one of the RESTful constraints (hypermedia describing what’s possible), as it may require the agent to have a priori knowledge of the query language and data model. The best you can do is have your media type include well-known hyperlinks to identify the query language and the data model that the interface is fronting.

    The point was “that’s OK!”, but just be aware of the tradeoff involved, that you’re putting a bigger burden on the client than if you just described in a more verbose hypertext format “here are the fields”, “here are the operands”, “here’s how to add predicates”, etc. There is a different sort of coupling involved. An analogy would be providing a full Query API , ala Oracle TopLink or Apple’s Core Data, to a developer vs. giving a programmer a JDBC Statement and DatabaseMetaData object and asking them to go to town with SQL.

    WRT Collection: Yes. A controller endpoint, in general, isn’t even a tradeoff! Define it in WSDL 2.0 if you want ;) So long as important resources that it can manipulate have URIs that you can do things with independently of the controller.

    WRT DELETE: Hopefully HTTPbis cleans this up. RFC 2646 claims client can’t assume *anything* that the server might do with the resource beyond the scope of the DELETE request. (i.e. “The client cannot be guaranteed that the operation has been carried out, even if the status code returned from the origin server indicates that the action has been completed successfully.”) This pushes the buck to the media type to constrain what DELETE implies on one of its hyperlinks.

  28. Umit:

    thank you for providing this reference. This is certainly one of the better work I have seen on REST. I have done a search on “version” to make sure I was not missing anything and to the best of my knowledge, the thesis only speaks about version instances, not version of the “business logic” (or whatever name people want to use). So I’ll stand by what I say. REST is unable to deliver a versioning strategy which is for me a non starter unless your community of service consumers is so dedicated that they will update their clients for every change you make (e.g. Google, Yahoo, Amazon…). In the enterprise you need a forwards compatible versioning strategy because projects don’t have the budget to update service consumers for every changes of the service implementation or contract. Operationally, the enterprise doesn’t have budgets to operates 15 versions of the same service either. This is why versioning is so important to SOA and this is one of the major contributions of WS-* (and Dave Orchard -thanks Dave).

    Stu:

    It is disappointing that we still having pretty much the same discussions that we had 3 years ago. If REST was what you guys were claiming then, we would not be having these discussions…So I’ll let everyone make their own conclusions. I don’t expect much from the RESTafarians, this is just a question of character.

    I am glad to see that more and more people don’t see it as a “silver bullet” (by far). If we could have had “normal” discussions then we would have saved 3 years to our industry and probably avoid RESTless disasters like Microsoft Astoria or OMG moments like Tim’s Slow REST. But whatever, ego and politics are more important than reason, this is just the way things are, not just in our Industry. I stand behind pretty much everything I wrote (loose coupling, actions/events, bi-directional interfaces, assemblies…). It is clear today that a lot of the people that make all these claims don’t have much experience building information systems. The latest CRUD frameworks like Spring Roo are the continuing illustration of this disconnect between the “pundits” and the real-world. That would need to change, but I don’t have much hope.

    just my $20 (inflation adjusted).

    JJ-

  29. Stu

    JJ – I’ve said it before, years ago, and I’ll repeat it here, since this seems to be an annual ritual. I actually agree with some of your work on object lifecycles and bi-directional interfaces. But people would listen to you and debate with you a lot more if they actually thought you were interested in making progress on improving REST, rather than questioning people’s character, motivation, or politics. Or repeating the same arguments over and over despite repeated attempts to provide evidence, logical argument, or counter example.

    You keep referring, for years now, to David Orchard’s writings on versioning as a great set of guidelines. Yet his writings were a draft finding while he was on the W3C TAG, which are… the maintainers and authors of the Web Architecture (http://www.w3.org/TR/webarch/) , and whose versioning finding was a revised look at some old notes on extensible languages by TBL and Dan Connolly back in the late 90’s…. in support of the Web Architecture. They are completely applicable to a RESTful architecture! There are also URI-based versioning schemes, which have their own set of tradeoffs vs. doing it at a language level. But, from what I read, you reject this approach outright. So I’m not sure what there is to discuss, you made your mind up years ago, and you continually pile on disincentive to constructive debate.

  30. Stu:

    you are funny…
    >> they actually thought you were interested in making progress on improving REST
    I don’t know many people in the industry who are more committed than I am to making progress. I have contributed countless ideas in the areas of SOA, BPM and MDE. I am not even committed to any vendor, hence to any particular technology.

    Who really cares about making progress? Is it the people that answer laconically or is it people that face the questions? That actually provide answers? So again, I repeat my question. How do you achieve Forwards Compatible Versioning mechanisms in REST? WS-* does it. It is essential to SOA and Composite solutions. It is essential to scaling the number of consumer of a service.

    Look at your answer:
    >> They are completely applicable to a RESTful architecture!
    No they are not, REST couples access and identity, REST has the wrong boundary and does not even have the concept of a “service”. A contract is essential to the versioning scheme. So, I am sorry say Stu -I also have a lot of respect for you- but here, on that critical question, you are no better than a Paul Downey or so many other RESTafarians that don’t know what they are talking about.

    So you can say whatever you want about me, but at the end of the day you are not giving answers to key questions. You are just asking like so many others “trust me, take this powerful medicine, and you’ll be cured…”. That does not work with me. This is not what I call progress.

    This is regression. Look at what happened in the last 3 years: you guys have successfully CRUDed the industry. Dave (Microsoft) Chappell claims SOA is a failure except for “Data Services”, even Umit, a co-author of the J2EE spec cornered herself to CRUD from an RIA client. What a regression ! what a step back ! what a terrible direction for large scale service consumption and for reuse. What a terrible factoring, even in the 60s people were doing better.

    All I am asking is no hand waving -that costs too much to our industry, it kills projects and companies- just tell me how you achieve Forwards Compatible Versioning in a RESTful way (i.e. changing the contract of the service provider without impacting existing consumer). The answer does not fit in one paragraph. Point me to an article, a thesis, a blog post, or tell me you don’t know.

    My claim -this is just a claim- is that it is not possible due to REST itself.

  31. Stu

    OK, for the sake of William’s sanity, I’ll take this to my blog. Later this weekend.

  32. Google gives these urls under “rest versioning”
    http://barelyenough.org/blog/2008/05/versioning-rest-web-services/
    http://barelyenough.org/blog/2008/05/versioning-rest-web-services-tricks-and-tips/

    I think versioning is just another indirection of resources if I know what I am talking about.

  33. Here is another discussion between Subbu and Pete Williams on the topic: http://www.subbu.org/blog/2009/12/media-types-and-plumbing

    >> another indirection of resources
    How do you achieve that when identity and access are coupled?

    JJ-

  34. Pingback: This Week in REST – Volume 5 (Feb 22 2010 – Feb 28 2010) « This week in REST

  35. Pingback: William Vambenepe — Two versions of a protocol is one too many

  36. JJ: “It seems that there is a disconnect. You seem to not understand (kind of) the problem that I am referencing. I am talking about versioning the business logic behind a RESTful call (GET, POST, PUT, DELETE), not the resources themselves.”

    If clients are driven by hypermedia, and interacting/exchanging representations by means of discovery (i.e. the system conforms to the hypermedia constraint/HATEOAS). Why on earth would the business logic behind the interface be of any meaningful concern to the client?

    With all due respect, you seem to not understand The Solution.