If clients are driven by hypermedia, and interacting/exchanging representations by means of discovery (i.e. the system conforms to the hypermedia constraint/HATEOAS). Why on earth would the business logic behind the interface be of any meaningful concern to the client?

With all due respect, you seem to not understand The Solution.

>> another indirection of resources
How do you achieve that when identity and access are coupled?

JJ-

I think versioning is just another indirection of resources if I know what I am talking about.

you are funny…
>> they actually thought you were interested in making progress on improving REST
I don’t know many people in the industry who are more committed than I am to making progress. I have contributed countless ideas in the areas of SOA, BPM and MDE. I am not even committed to any vendor, hence to any particular technology.

Who really cares about making progress? Is it the people that answer laconically or is it people that face the questions? That actually provide answers? So again, I repeat my question. How do you achieve Forwards Compatible Versioning mechanisms in REST? WS-* does it. It is essential to SOA and Composite solutions. It is essential to scaling the number of consumer of a service.

Look at your answer:
>> They are completely applicable to a RESTful architecture!
No they are not, REST couples access and identity, REST has the wrong boundary and does not even have the concept of a “service”. A contract is essential to the versioning scheme. So, I am sorry say Stu -I also have a lot of respect for you- but here, on that critical question, you are no better than a Paul Downey or so many other RESTafarians that don’t know what they are talking about.

So you can say whatever you want about me, but at the end of the day you are not giving answers to key questions. You are just asking like so many others “trust me, take this powerful medicine, and you’ll be cured…”. That does not work with me. This is not what I call progress.

This is regression. Look at what happened in the last 3 years: you guys have successfully CRUDed the industry. Dave (Microsoft) Chappell claims SOA is a failure except for “Data Services”, even Umit, a co-author of the J2EE spec cornered herself to CRUD from an RIA client. What a regression ! what a step back ! what a terrible direction for large scale service consumption and for reuse. What a terrible factoring, even in the 60s people were doing better.

All I am asking is no hand waving -that costs too much to our industry, it kills projects and companies- just tell me how you achieve Forwards Compatible Versioning in a RESTful way (i.e. changing the contract of the service provider without impacting existing consumer). The answer does not fit in one paragraph. Point me to an article, a thesis, a blog post, or tell me you don’t know.

My claim -this is just a claim- is that it is not possible due to REST itself.

You keep referring, for years now, to David Orchard’s writings on versioning as a great set of guidelines. Yet his writings were a draft finding while he was on the W3C TAG, which are… the maintainers and authors of the Web Architecture (http://www.w3.org/TR/webarch/) , and whose versioning finding was a revised look at some old notes on extensible languages by TBL and Dan Connolly back in the late 90′s…. in support of the Web Architecture. They are completely applicable to a RESTful architecture! There are also URI-based versioning schemes, which have their own set of tradeoffs vs. doing it at a language level. But, from what I read, you reject this approach outright. So I’m not sure what there is to discuss, you made your mind up years ago, and you continually pile on disincentive to constructive debate.

thank you for providing this reference. This is certainly one of the better work I have seen on REST. I have done a search on “version” to make sure I was not missing anything and to the best of my knowledge, the thesis only speaks about version instances, not version of the “business logic” (or whatever name people want to use). So I’ll stand by what I say. REST is unable to deliver a versioning strategy which is for me a non starter unless your community of service consumers is so dedicated that they will update their clients for every change you make (e.g. Google, Yahoo, Amazon…). In the enterprise you need a forwards compatible versioning strategy because projects don’t have the budget to update service consumers for every changes of the service implementation or contract. Operationally, the enterprise doesn’t have budgets to operates 15 versions of the same service either. This is why versioning is so important to SOA and this is one of the major contributions of WS-* (and Dave Orchard -thanks Dave).

Stu:

It is disappointing that we still having pretty much the same discussions that we had 3 years ago. If REST was what you guys were claiming then, we would not be having these discussions…So I’ll let everyone make their own conclusions. I don’t expect much from the RESTafarians, this is just a question of character.

I am glad to see that more and more people don’t see it as a “silver bullet” (by far). If we could have had “normal” discussions then we would have saved 3 years to our industry and probably avoid RESTless disasters like Microsoft Astoria or OMG moments like Tim’s Slow REST. But whatever, ego and politics are more important than reason, this is just the way things are, not just in our Industry. I stand behind pretty much everything I wrote (loose coupling, actions/events, bi-directional interfaces, assemblies…). It is clear today that a lot of the people that make all these claims don’t have much experience building information systems. The latest CRUD frameworks like Spring Roo are the continuing illustration of this disconnect between the “pundits” and the real-world. That would need to change, but I don’t have much hope.

just my $20 (inflation adjusted).

JJ-

WRT “polluting the resource model”, I think the root of this concern is how REST (hypertext, really) combines control and content into the same plane, whereas many other architectures separate the control plane out. For example, The tradeoff when you do this, is that the content itself directs the control plane (aka. “data-oriented” rather than “process-oriented”). When we’re so used to having data in its own world and process in its own world, it can be strange to see them brought back together.

WRT modelling actions in progress, it seems to me that evolution of long-running operations vs. short-running operations is a fairly universal tradeoff across architectures. I’m not aware of an architecture style that isolates my domain model from this sort of thing so that I can freely switch between short & long-lived with no impact to the model itself … unless I *assume* everything is long-lived and provide some mechanism to hide details for the short-lived case. This is the message-oriented middleware (MOM or JMS) way, in essence. It is built around the long-running case, even though I *could* just configure a transient, in-memory, non-persistent queue and make it look to my programmers as a regular procedure call.

The fundamental problem is whether the action itself is a datum worth tracking for a period of time or not. If not, then it’s not in my data model, if so, then it is. The tradeoff with REST is that the data model is visible, with MOM it’s hidden in the bowels of the message server. There were plenty of similar debates over the use of MOM vs. a database for persistent queueing.

WRT query, looking back, there are two ways to interpret the word “RESTful”. One is “approved by the REST gods”, the other is “relaxing some of the constraints, usually of the uniform interface”. I meant the latter, of course. A query interface with a query language is not really taking advantage of one of the RESTful constraints (hypermedia describing what’s possible), as it may require the agent to have a priori knowledge of the query language and data model. The best you can do is have your media type include well-known hyperlinks to identify the query language and the data model that the interface is fronting.

The point was “that’s OK!”, but just be aware of the tradeoff involved, that you’re putting a bigger burden on the client than if you just described in a more verbose hypertext format “here are the fields”, “here are the operands”, “here’s how to add predicates”, etc. There is a different sort of coupling involved. An analogy would be providing a full Query API , ala Oracle TopLink or Apple’s Core Data, to a developer vs. giving a programmer a JDBC Statement and DatabaseMetaData object and asking them to go to town with SQL.

WRT Collection: Yes. A controller endpoint, in general, isn’t even a tradeoff! Define it in WSDL 2.0 if you want So long as important resources that it can manipulate have URIs that you can do things with independently of the controller.

WRT DELETE: Hopefully HTTPbis cleans this up. RFC 2646 claims client can’t assume *anything* that the server might do with the resource beyond the scope of the DELETE request. (i.e. “The client cannot be guaranteed that the operation has been carried out, even if the status code returned from the origin server indicates that the action has been completed successfully.”) This pushes the buck to the media type to constrain what DELETE implies on one of its hyperlinks.

There are similar religious debates occuring on the data storage world – NOSQL, SQL, KV stores, graph databases, columnar stores, and so on. The reality is that a well crafted architecture can blend many of these technologies. I’ve found it just as important to consider the “consumability” of your functionality as it is to consider the plumbing, with an eye on understandable semantics, self-documentation, extensibility, and support for a broad range of human/software “consumers”. Just my $0.02, after taxes.

– Jan’s thesis: http://www.scribd.com/doc/2218300/WebData-Defnition-of-a-Middleware-for-Exposing-and-Accessing-Objectoriented-Domain-Models-as-Web-Resources

– Our presentation on RIA. This does not provide the details on the protocol, but puts you into the frame of reference why/where we were using RESTful services. That is linked to my profile in linked in and probably most of you have seen it already.

http://tinyurl.com/ycfna97

JJ, we did not address business logic behind resource’s and their versioning. I do not claim that this approach addresses that issue or it is tied to the protocol. However, I would imagine that as business logic if it was tied to a resource were to change on the backend, it would be “logical” to expose a different version of the resource.

Why are REST discussions are always RESTless.

You guys better be careful — it’s starting to smell like rest-discuss in here…

One point: my understanding of DELETE from RFC2616 is the intention is to remove the resource, or make it inaccessible. I find it useful to consider an intermediary such as a proxy who on seeing a DELETE would discard any cached version of a representation of a resource, but would still honor a subsequent GET to the URI. So whilst it’s possible for the server to continue returning zombie copies of the DELETEd resource, it’s not really what DELETE means, and my suggestion is to POST/PUT a change of state to “deleted” and keep a record of the resource is usually what most people expect.

I do find it hard to talk about “business logic” because for me that’s invariably something of an oxymoron. It’s a phrase, which along with the notion of machine readable contracts I get by these days without hearing or needing use. As for compatible evolution, three simple rules have served us all very well over the years:

1) have sensible defaults for missing values for backwards compatibility.
2) design formats with ignorable extra values for forwards compatibility.
3) if you change the meaning of stuff, or the processing in a way the client doesn’t expect, then it’s a breaking change, so use a different interface.

kthxbai!

WRT to “polluting the resource model with artifacts that have little to do with the business at hand” I indeed think this is something we should strive to avoid. It doesn’t mean that I am not open to compromise. Allowing some amount of abstraction leakage might be the pragmatic choice. We do this all the time, as you point out, because perfect abstraction layering (or something close to it) usually comes at a huge cost. But we have to realize that there is also a cost to allowing this pollution/contamination/leakage. So I feel justified in listing this as a drawback (not a deal-breaker).

WRT to modeling actions in progress, I don’t have a problem with modeling a server in the process of being instantiated, or an application in the process of being deployed, as these are representations of a true aspect of the domain I am modeling. My issue is with the idea that any operation that may take some time to execute (e.g. a long-running query) fits the same model. To me, there is a difference between actions in progress that represent the realities of the domain and actions in progress that represent the technical limitations of my infrastructure. What if tomorrow I upgrade from SQL Server to Oracle DB and my queries become blazingly fast? Am I then stuck w/ model elements that are vestiges of my old infrastructure? I’d rather not have those in the model, so I only have to deal w/ model changes when the entities in the domain I manage actually change, not when my management infrastructure changes.

For the same reason, I have no problem w/ modeling a “subscription” but I want to keep any info about how the delivery takes place out of the model. That should just be, from the model perspective, a self-described data element (e.g. a URL w/ whatever appropriate scheme).

WRT to query, you and I have had this debate before, in the comments for this post. There, you agree with my statement that

“[A query] can return results that open a world of RESTful requests to you, but the query invocation itself is not RESTful. And that’s OK”.

You add that “[A query] could generate hypermedia docs, even if the query invocation itself isn’t RESTful. That’s OK, and a great example of this in the SemWeb world would be a SPARQL endpoint, something that really does tunnel a query interface through HTTP”.

So what are we arguing about?

I am not trying to play “gotcha”, just showing that I really value and remember the great comments deposited on this blog…

WRT to enumeration, my fear (maybe unjustified) is that it’s now the REST approach that is flirting w/ over-engineering (e.g. links for enumeration within a resource rather than for linking actual resources as in RFC 5005). Versus the simpler URL-parameter approach that Paul suggests but is less RESTful than MNot’s RFC.

Collection: what resource do I send my “reboot instances 1, 2, 5 and 8″ message to? Do I need to model a “bag” resource that contains them all? Do I create a resource specifically for these instances and this operation? Do I keep around such large “bag” resources and pass the ids of the server to reboot as a parameter? Compare this to the simpler task of sending a “reboot instances 1, 2, 5 and 8″ to a shadowy, not-part-of-the-resource-model controller endpoint that can do things for me. In effect, for some operations I want to be able to address explicit resources in a RESTful way, but for some others (query, collection operations) I want to be able to send them “in the Cloud” to be handled by some invisible infrastructure controller. And of course there should be a tie between these two sets of invocations (e.g. the use of hypermedia in the responses). I should probably do a full blog entry on this dichotomy.

WRT to DELETE I think this becomes again a bit complex and variable. Paul Downey, for example, seemed to think in his reply post that DELETE indeed means you only return 410 Gone on it (but you could still stuff in some info in the 410 reply). It might be easier to provide a simple way to access deleted resources than to finesse the per-media-type semantics of DELETE. Though on that one I could actually be convinced to find a way to make DELETE work as we want.

it seems that there is a disconnect. You seem to not understand (kind of) the problem that I am referencing. I am talking about versioning the business logic behind a RESTful call (GET, POST, PUT, DELETE), not the resources themselves. I understand that versioning resources has been solved decades ago. Unless you believe that REST is directly wired to the metal, you have lots of code behind a POST or a PUT for instance. How do you version that code?

I said “kind of” earlier because you explain “Compatible evolution in practice is far simpler on the Web, in particular using HTML and JSON, where there is no notion of a shared model or formal contract”, yet you seem to not understand what “compatible evolution” means. How can you be “compatible” without a “contract”. You just “hope” for compatibility? how can you test, certify, assert compability? without an agreed upon contract and rules for compatibility (described in the paper that I referenced earlier).

So again, I would highly recommend you take all the “emotion” out of the question and look at the problems in their entirety while avoiding the laconic “XXX solves your problem”.

WS-* had the merit of kind of supporting loose coupling and versioning which are both impossible to achieve in REST. We are throwing away tremendous amounts of value simply because some people don’t know how to use XML or XSD, let alone WSDL. That is sad.

JJ-

hi, I am neither anti-REST nor a Pundit. I am not a brand, nor wish to have “followers”, I don’t use my name as an argument. I use arguments and if I am wrong, I’ll be the first one to admit that I am wrong.

I am not anti-REST, I am anti “everything is better , solved, wonderful, and filled with puppies world”. I fully support approaches such as the one from Subbu who simply says, if you are going to use HTTP in one way or another, here is how you should use it. I fully support Roy’s views on REST and REST as the foundation for the Web. Now when people claim that REST is “middleware” or worse, REST is a “programming model” (that comes with its own QL, asynchrony, eventing mechanisms, ***) far more superior to anything that already exist that’s where I … (fill in the blanks).

Now when someone makes a claim such as “versioning in REST [can be achieved with] WebDAV/svn or most any DVCS is pretty RESTful compared with the impossibility of evolving SOAP messages” and that someone offers no substantiation, I feel I have the right to call him both stupid and ignorant. He/She has always the choice to do like Umit did to substantiate his or her claims.

Our industry has been made so sick by its punditocracy (of which I am not part of, and will never be part of) that we have now the characteristics of the pharmaceutical industry (without the FDA and double-blind processes). The software industry is now in the business of inventing “remedies” to all the diseases it created by “following” without “questioning”. You get the picture, right? can you imagine what the pharmaceutical industry would look like if any one could claim whatever they wanted without “questioning”.

So I repeat, there is no documented versioning mechanism in REST despite what the punditocrats claim laconically. Even what Umit has written doesn’t qualify as an answer, though she provides a lot of details about the context in which she made the claims. I believe her context is way too limited to define a general versioning strategy (as most of the business logic is in the RIA client, which is an anti-SOA pattern and a common problem of siloed solutions).

The RESTafarian punditocrats could have at least the decency to revisit their claims from 2007 about “uniform interface” for instance (in the light of RESTfulie) and come forward to express how wrong they were about claim an interface to a resource is uniform. For me, until versioning is solved (and I don’t see how), for me the (other) REST (not the one of Roy) is a non starter in the Enterprise/IT. It is, so far, just a collection of recipes to use HTTP here and there to solve tactical problems. I am still waiting for the examples of enterprise solutions built in a 100% RESTful way (ERP, CRM, PLM…).

Second, William, when you say “polluting the resource model with artifacts that have little to do with the business at hand”, I find it curious why you would think the resource model has to be 1:1 mapped with the business at hand? In what technology has that *ever* occurred? Physical RDBMS design? SOAP APIs? The closest *might* be pure object-oriented domain models persisted via an ODBMS or a sophisticated O/R mapper. How many of those have you seen in the wild?

On particular topics:

Long-lived operations: Yes, an action in progress resource makes sense, as it would in a website. Or in a database with a jobs table. Or an EPR for a subscription to an event sink in WS-Eventing.

Query: There are two styles of query here. One is where the query options are described in machine-readable hypertext and POSTed back, as how we do queries today with a Form. The other is where the query language required is indicated in hypertext, along with a description of the data model, and the agent POSTs the query language back, which is how, say SPARQL works. I’m not sure where the impracticality comes in.

Events: This problem exists with any eventing mechanism out there, so I’m not sure what the problem is here.

Enumeration: This is not the problem of GET. That would be like saying it’s WS-Transfer’s problem, and we don’t need WS-Enumeration. This is the problem of a media type that can handle enumeration. RFC 5005 is one solution. OpenSearch has another.

Filtering: Design a model that allows you to filter results through a query?

Collections: I’m curious. Why can’t you do exactly what you described? Under what RESTful tenant?

The afterlife: DELETE expresses intent. It doesn’t mean it has to go away, (e.g. 410 Gone) afterwards. You can DELETE something, then GET it, and something might still be there for posterity. How this works is up to how the media type you’re using defines the semantics of DELETE on the particular hyperlink the agent wants to DELETE.

This whole post has a flavour of “REST doesn’t wash my socks”, and of course it doesn’t, it’s like saying “pub/sub eventing doesn’t wash my socks”. You are holding WebArch to some kind of abstract standard that you don’t even hold WS-* to, meaning it’s supposed to solve all my problems without requiring me to engineer solutions.

The problem with REST, and here’s where I agree with JJ, it’s that it’s not this “everything is better , solved, wonderful, and filled with puppies” world that some make it out to be. A sign that you’re dealing with one of those foks is their instance on using PUT for everything and avoiding POST, because, you know, that would require more work in defining a media type that uses POST appropriate. There’s a lot of work to be done, and a lot of standard media types that would be very helpful the machine agents that are…. missing. Atompub and its extensions are only the beginning, and it’s a bit frightening that it’s taken this long to make so little progress. But that’s mainly because people are building local solutions to their problems, can kind of sick of standards organizations.