Category Archives: Tech

Schema-based XPath tool

Most XML editors offer an XPath tool that allows one to test and fine-tune XPath expressions by running them against XML documents. Very helpful but also potentially very deceptive. With such a tool it is very easy to convince oneself that an XPath expression is correct after running it against a few instance documents. And a month later the application behave erratically (in many cases it probably won’t break it will execute the request on the wrong element which is worst) because the XPath expression is ran on a different document and what it returns is not what the programmer had in mind. This is especially likely to occur as people use and abuse shortcuts such as “//” and ignore namespaces.

What we need is an XPath tool that can run not only run the XPath against an instance document but can also run it against a schema. In the later case, the tool would flag any portion of the schema that can possibly correspond to a node in the resulting nodeset. It would force programmers to realize that their //bar can match the /foo/bar that they want to reach but it could also match something that falls under the xsd:any at the end of the schema. And the programmer has to deal with that.

1 Comment

Filed under Everything, Implementation, Tech

Humble Architecture

In many respects, the principles of Service-Oriented Architecture (SOA) can be summarized as “be humble”. “Service” comes from “servus”, Latin for “slave”. It doesn’t get any more humble.

More practically, this means that the key things to keep in mind when creating a service, is that you are not at the center of the universe, that you don’t know who is going to consume your service, that you don’t know what they are going to do with it, that you are not necessarily the one who can make the best use of the information you have access to and that you should be willing to share it with others openly (instead of the all-too familiar syndrome where everyone wants to consume other people’s services but no-one sees the need to expose themselves as services because they think they “own” the connection to the human or they “own” the business process). You also shouldn’t assume that some human needs to come to you and ask for permission to use your service but instead you should provide machine-readable descriptions of it as well as quality documentation. And don’t assume that everyone speaks the same language you speak. In case of doubt in designing a service-oriented system, ask yourself “what would a slave do?”.

Focused, standard-based services are humble. Portlets are humble. RSS feeds are humble. Giant software suites and all-encompassing frameworks are not humble.

Successful open source projects are humble almost by definition. Large software companies rarely have humility genes in their DNA, unless it’s been beaten into them by customers.

2 Comments

Filed under Everything, Tech

Updating an EPR

The question recently came back on the WS-Addressing mailing list of whether Reference Parameters can/should be used as the SOAP equivalent of cookies. Something more along the lines of session management than addressing. See Peter Hendry’s email for a clear description of his use case. The use case is reasonable but I don’t think this is what WS-Addressing is really for as I explain in bullet #3 of this post. What interested me more was the response that came from Conor Cahill and his statement that AOL is implementing an “EndpointReferenceUpdate” element that can be returned in the response to tell the sender to update the EPR. I am not fond of this as a mechanism for session management, but I can see one important benefit of this mechanism: getting hold of a “good” EPR for more efficient addressing. Here is an example application:

Imagine a Web services that represents the management interface of a business process engine. That Web service provides access to all the currently running business process instances in the engine (think Service Group if you’re into WSRF). Imagine that this Web service supports a SOAP header called “target” and that header is defined to contain an XPath statement. When receiving a message containing a “target” header, the Web service will look for the (for the sake of simplicity let’s assume there can only be one) business process instance for which this XPath statement returns “true” when evaluated on the XML representation of the state of the business process instance. And the Web service will then interpret the message to be targeted at that business process instance. This is somewhat similar to WS-Management’s “SelectorSet”. A sender can use this mechanism to address a specific business process instance based on the characteristics of that instance (side note: whether the sender understands and builds this header itself or whether it gets it as a Reference Parameter from an EPR is orthogonal). But this can be a very expensive dispatching mechanism. The basic implementation would require the Web service to evaluate an XPath statement on each and every business process instance state document. Far from optimal. This is where Conor’s “EndpointReferenceUpdate” can come in handy. After doing once the XPath evaluation work of finding out which business process instance the sender wants to address, the Web service can return a more optimized EPR to be used to address that instance, one that is a lot easier to dispatch on. This kind of scenario is a lot more relevant in my perspective to the work of the WS-Addressing working group than the session example.

An important consequence of a mechanism such as “EndpointReferenceUpdate” is that it makes it critical that the Web service be able to tell which SOAP headers are in the message as a result of being in the EPR used by the sender and which ones were added by the sender on purpose. For example, if a SOAP message comes in with headers “a”, “b” and “c” and the Web service assumes that “a” and “b” were in the EPR and “c” was added by the invoker, then the new EPR returned as part of “EndpointReferenceUpdate” will only be a replacement for “a” and “b” and the Web service will still expect “c” to be added by the sender. But if in fact “c” also came from a reference parameter in the EPR used by the sender then follow-up messages will be incomplete. This puts more stress and responsibilities on the already weak @isReferenceParameter attribute. And, by encouraging people to accept EPRs from more and more sources, it puts EPR consumers are even greater risk for the problems described in bullet (1) of this objection.

2 Comments

Filed under Everything, Security, Standards, Tech

A map to federated IT model repositories

Using scissors and tape, one can stitch street maps and road maps together to obtain an aggregated map showing how to go from downtown Palo Alto to downtown San Francisco. The equivalent in IT management is to stitch together different model repositories by federating them, as a way to get a complete view of an IT system of interest. As we go about creating the infrastructure for model federation, there is a lot to be learned from the evolution of street maps.

Let’s go back to paper maps for a minute. A map of the Bay Area will tell me what highways to take to go from Palo Alto to SF. But it won’t help me get from a specific house in Palo Alto to the highway and once in SF it won’t help me get from the highway to a specific restaurant. For this, I need to find maps of downtown Palo Alto and downtown SF and somehow stitch the three maps together for an end to end view. Of course all these maps have different orientations, different scales, partial overlap, different legends, etc. Compare this to using Google maps which covers the entire itinerary and allows the user to zoom in and out at will.

Let’s now go back to IT management. In order to make IT systems more adaptable, the level of automation in their management must drastically increase. This requires simplification. Trying to capture all the complexity of a system in one automation point is neither scalable nor maintainable. But one cannot simply wave a wand and make a system simpler. The basic building blocks of IT are not getting simpler: the number of transistors on a chip is going up, the number of lines of code in an application is going up, the number of data items in a customer record is going up. Literal simplification would be going back to mechanical calculators and paper records… What I really mean by simplification is decomposing the system into decision points (or control points) that process information and take action at a certain level of granularity. For example, an “employee provisioning” control point is written in terms of “mail account provisioning” and “payroll addition”, not in terms of “increasing size of a DB table”. That’s simplification. Of course, someone needs to worry about allocating enough space in the database. There is another control point at that lower level of granularity. The challenge in front of us is to find a way to seamlessly integrate the models at these different levels of granularity. Because they are obviously linked. The performance and reliability of the “employee provisioning” service is affected by the performance and reliability of the database. Management services need to be able to navigate across these models. We need to do this in a way inspired by Google Maps, not by stitching paper maps. Let’s use the difference between these two types of maps to explore the requirements of infrastructure for IT models federation.

Right level of granularity

The publishers of a paper map decide, based on space constraints, which streets are shown. With Google Maps, as you zoom in and out smaller streets show up and disappear. Similarly, an IT model should be exposed in a way that allows the consumer to decide what level of granularity is presented.

Machine-readable

Paper maps are for people, Google Maps can be used by people and programs. IT models must be exposed in a way that doesn’t assume a human sitting in front of a console is the consumer of the information.

Open to metadata and additional info

To add information to a paper map, you have to retrieve the information, find out where on the map it belongs and manually add it there. Google map lets you overlay any information directly on top of the map (see Housingmaps.com). Similarly, IT model federation requires the ability to link metadata and extra model information about model elements to the representation of the model, even if that information resides outside the model repository.

Standards-based

Google provides documentation for its maps service. It’s not a standard, but at least it’s documented and publicly accessible. Presumably they are not going to sue their users for patent violation. Time will tell whether this is good enough for the mapping world. In the IT management world, this will not be enough. Customers demand real standards to protect their investment, speed up deployment and prevent unneeded integration costs. Vendors need it as protection (however imperfect) against patent threats, as a way to focus their energy on value-added products rather than plumbing and just because smart customers demand it.

Seamless integration

I don’t know if Google gets all its mapping information from one source or from several, and I don’t need to know it. As I move North, South, East, West and zoom in and out, it is a seamless experience. The same needs to be true in the way federated models are exposed. The framework through which this takes place should provide seamless integration across sources. And simplify as much as possible discovery of the right source for the information needed.

Support for different metamodels

Not all maps use the same classification and legend. Similarly, not all models repositories use the same meta-model. Two meta-models might have the notion of “owner” of a resource but call it differently and provide different information about the owner. Seamless integration requires support for model bridging.

Searchable

Federated models repositories need to be efficiently searchable.

Up to date

Paper maps age quickly. Google Maps is more likely (but not guaranteed) to be up to date. Federated models must be as close a representation of the real state of the system as possible.

Secure

As you are composing information from different sources, the seamless navigation among these resources needs to be matched by similar seamless integration in the way the access is secured, using security federation.

Note 1: When I talk about navigating “models” in this entry, I am referring to an instance model that describes a system. For example, such a “model” can be a set of applications along with the containers in which they live, the OS these containers run on and the servers that host them. That’s one “model”. If the information is distributed among a set of MBean servers, CMOM, etc, then this is a federated model. I know some people don’t call this a “model” and I am not married to this word. Based on the analogy used in this entry, “system map” and “federated system map” would work just as well.

Note 2: This entry corresponds to the presentation I gave when participating in a panel (which I also moderated) on “Quality of Manageability of Web Services” at the IEEE ICWS 2005 conference in Orlando last week. The other speakers were Dr. Hemant Jain (UW Milwaukee), Dr. Hai Jin (Huazhong University of Science and Technology), Heather Kreger (IBM), Dr. Geng Lin (Cisco). Unfortunately, the presentation was made quite challenging when (1) the microphone stopped working (it was in a large ballroom), (2) a rainstorm had us compete with the sound of thunder, (3) torrential rain started to fall on the roof of our one-story building, turning the room into a resonance box and, to top it off, (4) the power went off completely in the entire hotel leaving me to try to continue talking by the light of the laptop screen and the emergency exit lights…. With all this plus time constraints, I am not sure I did a good job making my point clear. This entry hopefully does a better job than the presentation. The conference was quite interesting. In addition to the panel I also presented a co-authored paper based on an HP Lab project. The paper is titled “Dealing with Scale and Adaptation of Global Web Services Management”. The conference also allowed me to finally meet Steve Loughran face to face. Congrats to Steve and Ed Smith for being awarded the “Best paper” award for “Rethinking the Java SOAP stack“, also known as “the Alpine paper”. When a papers gets a nickname you know it is having an impact…

1 Comment

Filed under Everything, Research, Tech

EPR redefining the difference between SOAP body and SOAP header

The use of WS-Addressing EPRs is redefining the difference between SOAP body and SOAP headers. The way the SOAP spec looks at it, the difference is that a header element can be targeted at an intermediary, while the body is meant only for the ultimate receiver. But very often, contract designers seem to decide what to put in headers versus body less based on SOAP intermediaries than on the ability to create EPRs. Basically, parts of the message are put in headers just so that an EPR can be built that constrains that message element. To the point sometimes of putting the entire content of the message in headers and leaving an empty body (as Gudge points out and as several specs from his company do). And to the contrary, a wary contract designer might very well put info in the body rather than a header just for the sake of “protecting” it form being hard-coded in an EPR (the contract requires that the sender understands this element, it can’t be sent just because “an EPR told me to”).

This brings up the question: rather than twisting SOAP messages to accommodate the EPR mechanism, should the EPR mechanism be made more flexible in the way it constrains the content of a SOAP message?

Comments Off on EPR redefining the difference between SOAP body and SOAP header

Filed under Everything, Standards, Tech

Spreading the word of SOA and SOA management

Over the last couple days, a few articles came up that help explain HP’s vision for Management of the Adaptive Enterprise, so here are the links.

Yesterday, Mark Potts published an article describing the value of SOA for enterprises and more specifically the management aspects of SOA (security, life cycle and configuration, management of infrastructure services and business services, governance, etc). BTW, the SOA practice from HP Consulting and Integration that Mark refers to at the end of his article is what I mentioned in my previous post.

Another interesting article is Alan Weissberger’s entusiastic report from GGF 14. Alan follows GGF and related OASIS activities very closely, doesn’t fall for fluff and is not easily impressed so this a testimony to the great work that Heather, Bryan, Bill and Barry did there, presenting a WSDM deep dive, the HP/IBM WSDM demos (which they also showed at IEEE ICAC in Seattle) and talking about the recently released HP/IBM/CA roadmap for management using Web services. These four should call themselves “Heather and the Bs” or “HB3” for short if they keep touring the world showing their cool demos. Can’t wait to see them at the Shoreline Amphitheatre. Of course, Alan’s positive comments also and mainly come out of all the hard technical work that lead to this successful GGF14, including the OGSA WSRF Basic Profile.

Two more articles to finish, both about the HP/IBM/CA roadmap. I talked to the journalists for both of these articles, one form ComputerWorld and one from the Computer Business Review.

Four good articles in two days, it is very encouraging to see how the understanding of how we are unleashing the power of SOAs through adaptive management is growing. This is what the roadmap is all about, explaining the objectives to people and inviting them on board.

Comments Off on Spreading the word of SOA and SOA management

Filed under Business, Everything, Tech

So you want to build an EPR?

EPR (Endpoint References, from WS-Addressing) are a shiny and exciting toy. But a sharp one too. So here is my contribution to try to prevent fingers from being cut and eyes from being poked out.

So far I have seen EPRs used for five main reasons, not all of them very inspired:

1) “Dispatching on URIs is not cool”

Some tools make it hard to dispatch on URI. As a result, when you have many instances of the same service, it is easier to write the service if the instance ID is in the message rather than in the endpoint URI. Fix the tools? Nah, let’s modify the messages instead. I guess that’s what happens when tool vendors drive the standards, you see specifications that fit the tools rather than the contrary. So EPRs are used to put information that should be in the URI in headers instead. REST-heads see this as a capital crime. I am not convinced it is so harmful in practice, but it is definitely not a satisfying justification for EPRs.

2) “I don’t want to send a WSDL doc around for just the endpoint URI”

People seem to have this notion that the WSDL is a “big and static” document and the EPR is a “small and dynamic” document. But WSDL was designed to allow design-time and run-time elements to be separated if needed. If all you want to send around is the URI at which the service is available, you can just send the URI. Or, if you want it wrapped, why not send a soap:address element (assuming the binding is well-known). After all, in many cases EPRs don’t contain the optional service element and its port attribute. If the binding is not known and you want to specify it, send a around a wsdl:port element which contains the soap:address as well as the QName of the binding. And if you want to be able to include several ports (for example to offer multiple transports) or use the wsdl:import mechanism to point to the binding and portType, then ship around a simplified wsdl:descriptions with only one service that itself contains the port(s) (if I remember correctly, WS-MessageDelivery tried to formalize this approach by calling a WSRef a wsdl:service element where all the ports use the same portType). And you can hang metadata off of a service element just as well as off of an EPR.

For some reason people are happy sending an EPR that contains only the address of the endpoint but not comfortable with sending a piece of WSDL of the same size that says the same thing. Again, not a huge deal now that people seem to have settled on using EPRs rather than service elements, but clearly not a satisfying justification for inventing EPRs in the first place.

3) “I can manage contexts without thinking about it”

Dynamically generated EPRs can be used as a replacement for an explicit context mechanism, such as those provided by WS-Context and WS-Coordination. By using EPRs for this, you save yourself the expense of supporting yet-another-spec. What do you loose? This paper gives you a detailed answer (it focuses on comparing EPRs to WS-Context rather than WS-Coordination for pretty obvious reasons, but I assume that on a purely technical level the authors would also recommend WS-Coordination over EPRs, right Greg?). In a shorter and simplified way, my take on the reason why you want to be careful using dynamic EPRs for context is that by doing so you merge the context identifier on the one hand and the endpoint with which you use this context on the other hand into one entity. Once this is done you can’t reliably separate them and you loose potentially valuable information. For example, assume that your company buys from a bunch of suppliers and for each purchase you get an EPR that allows you to track the purchase as it is shipped. These EPRs are essentially one blob to you and the only way to know which one comes through FedEx versus UPS is to look at the address and try to guess based on the domain name. But you are at the mercy of any kind of redirection or load-balancing or other infrastructure reason that might modify the address. That’s not a problem if all you care about is checking the ETA on the shipment, each EPR gives you enough information to do that. But if you also want to consolidate the orders that UPS is delivering to you or if you read in the paper about a potential UPS drivers strike and want to see how it would impact you, it would be nice to have each shipment be an explicit context Id associated to a real service (UPS or FedEx), rather than a mix of both at the same time. This way you can also go to UPS.com, ask about your shipments and easily map each entry returned to an existing shipment you are tracking. With EPRs rather than explicit context you can’t do this without additional agreements.

The ironic thing is that the kind of mess one can get into by using dynamic EPRs too widely instead of explicit context is very similar in nature to the management problems HP OpenView software solves. Discovery of resources, building relationship trees, impact analysis, event correlation, etc. We do it by using both nicely-designed protocols/models (the clean way) and by using heuristics and other hacks when needed. We do what it takes to make sense of the customer’s system. So we could just as well help you manage your shipments even if they were modeled as EPRs (in this example). But we’d rather work on solving existing problems and open new possibilities than fix problems that can be avoided. And BTW using dynamic EPRs is not always bad. Explicit contexts are sometimes overkill. But keep in mind that you are loosing data by bundling the context with the endpoint. Actually, more than loosing data, you are loosing structure in your data. And these days the gold is less in the raw data than in its structure and the understanding you have of it.

4) “I use reference parameters to create new protocols, isn’t that cool!”

No it’s not. If you want to define a SOAP header, go ahead: define an XML element and then describe the semantics associated with this element when it appears as a SOAP header. But why oh why define it as a “reference parameter” (or “reference property” depending on your version of WS-A)? The whole point of an EPR is to be passed around. If you are going to build the SOAP message locally, you don’t need to first build an EPR and then deconstruct it to extract the reference parameters out of it and insert them as SOAP headers. Just build the SOAP message by putting in the SOAP headers you know are needed. If your tooling requires going through an EPR to build the SOAP message, fine, that’s your problem, but don’t force this view on people who may want to use your protocol. For example, one can argue for or against the value of WS-Management‘s System and SelectorSet as SOAP headers, but it doesn’t make sense to define those as reference parameters rather than just SOAP headers (readers of this blog already know that I am the editor of the WSDM MUWS OASIS standard with which WS-Management overlaps so go ahead and question my motives for picking on WS-Management). Once they are defined as SOAP headers, one can make the adventurous decision to hard-code them in EPRs and to send the EPRs to someone else. But that’s a completely orthogonal decision (and the topic of the fifth way EPRs are used – see below). But using EPRs to define protocols is definitely not a justification for EPRs and one would have a strong case to argue that it violates the opacity of reference parameters specified in WS-Addressing.

5) “Look what I can do by hard-coding headers!”

The whole point of reference parameters is to make people include elements that they don’t understand in their SOAP headers (I don’t buy the multi-protocol aspect of WS-Addressing, as far as I am concerned it’s a SOAP thing). This mechanism is designed to open a door to hacking. Both in the good sense of the term (hacking as a clever use of technology, such as displaying Craig’s list rental data on top of Google maps without Craig’s List or Google having to know about it), and in the bad sense of the term (getting things to happen that you should not be able to make happen). Here is an example of good use for reference parameters: if the Google search SOAP input message accepted a header that specifies what site to limit the search on (equivalent to adding “site:vambenepe.com” in the Google text box on Google.com), I could distribute to people an EPR to the vambenepe.com search service by just giving them an EPR pointing to the Google search service and adding a reference parameter that corresponds to the header instructing Google to limit the search to vambenepe.com.

Some believe this is inherently evil and should be stopped, as expressed in this formal objection. I think this is a useful mechanism (to be used rarely and carefully) and I would like to see it survive. But there are two risks associated with this mechanism that people need to understand.

The first risk is that EPRs allow people to trick others into making statements that they don’t know they are making. This is explained in the formal objection from Anish and friends as their problem #1 (“Safety and Security”) and I agree with their description. But I don’t agree with the proposed solutions as they prevent reference parameters to be treated by the service like any other SOAP header. Back last November I made an alternative proposal, using a wsa:CoverMyRearside element that would not have this drawback and I know other people have made similar proposals. In any case, this risk can and should be addressed by the working group before the specification becomes a Recommendation or people will stop accepting to process reference parameters after a few high-profile hacks. Reference parameters will become the ActiveX of SOAP.

The second risk is more subtle and that one cannot be addressed by the specification. It is the fragility that will result from applications that share too many assumptions. I get suspicious when someone gives me directions to their house with instructions such as “turn left after the blue van” or “turn right after the barking dog”, don’t you? “We’re the house after the green barn” is a little better but what if I want to re-use these directions a few years later. What’s the chance that the barn will be replaced or repainted? EPRs that contain reference parameters pose the same problem. Once you’ve sent the EPR, you don’t know how long it will be around, you don’t know who it will get forwarded to, you don’t know what the consumer will know. You need to spend at least as much efforts picking what data you use as a reference parameter (if anything) as you spend designing schemas and WSDL documents. If your organization is smart enough to have a process to validate schemas (and you need that), that same process should approve any element that is put in a reference parameter.

Or you’ll poke your eye out.

2 Comments

Filed under Everything, Implementation, Security, Standards, Tech

HP/IBM/CA roadmap white paper

HP, IBM and CA recently released a white paper describing how we see the different efforts in the area of management for the adaptive enterprise coming together and, more importantly, what else is needed to fulfill the vision. Being a co-author I am arguably more than a little biased, but I recommend the read as an explanatory map of the standards/specifications landscape, from the low levels of the Web services stack all the way up to model transformations and policy-based automated management: http://devresource.hp.com/drc/resources/muwsarch/index.jsp

Comments Off on HP/IBM/CA roadmap white paper

Filed under Articles, Everything, Standards, Tech

Someone is paying attention

It’s nice to see that, while most of the tech press seems happy to copy/paste from misleading press briefing documents rather than do any checking of their own, some analysts take a little bit more time to look through the smoke. So, when Gartner looks into the recent Microsoft/Sun announcement (see “Progress Report on Sun/Microsoft Initiative Lacks Substance”) their recommendation is to “view the latest Sun/Microsoft announcement as primarily public-relations-oriented”. Similar take from Jason Bloomberg from ZapThink who thinks that this “doesn’t do anything to contradict the fact that Microsoft is the big gorilla in this relationship”. And Forrester’s Randy Heffner (quoted in “Analysts Question Microsoft-Sun Alliance”) thinks that “Bottom line: Web services interoperability is not yet part of the picture”. Oh, and by the way “the WS-Management group has yet to come clean on how they will work with the WSDM standard approved by OASIS,” Heffner also says. “Again, WS-Management is still just a specification in the hands of vendors”. Very much so. But in PR-land everything looks different. As tech journalists write these articles including insight from analysts that contradict what the tech press reported a couple days earlier, I wonder if they ever think “hum, maybe I should be the one doing reality checks on the content of press releases rather than going around collecting quotes and then the analysts would focus on real in-depth analysis rather than just doing the basic debunking work…”

Comments Off on Someone is paying attention

Filed under Business, Everything, Standards, Tech

Reality check on Microsoft/Sun claims about single sign-on

This morning I learned that Microsoft and Sun had a public event where the CEOs reported on a year of working together. This is a follow-up to Greg Papadopoulos’ report on the progress of the “technical collaboration”. In that post, Greg told us about the amazing technical outcomes of the work between the two companies and, being very familiar with the specs he was referring to, I couldn’t help but point out that the result of the “technical collaboration” he was talking about looked a lot like Sun rubber-stamping a bunch of Microsoft specifications without much input from Sun engineers.

So when I heard this morning that the two companies were coming out publicly with the result of their work, I thought it would be fair for me to update my blog and include this information.

Plus, reading the press release and Greg’s Q&A session, it sounded pretty impressive and it would have been bad faith from my part to not acknowledge that indeed Greg actually had something to brag about, it just wasn’t yet public at the time. In effect, it sounded like they had found a way to make the Liberty Alliance specs and WS-Federation interoperate with one another.

From Greg’s Q&A: “In a nutshell, we resolved and aligned what Microsoft was trying to accomplish with Passport and the WS-Federation with what we’ve been doing with the Liberty Alliance. So, we’ve agreed upon a way to enable single sign-on to the Internet (whether through a .NET service or a Java Enterprise System service), and federate across those platforms based on service-level agreements and/or identity agreements between those services. That’s a major milestone.”

Yes Greg, it would have been. Except this is not what is delivered. The two specs that are supposed to support these claims are Web SSO MEX and Web SSO Interop Profile. Which are 14 and 9 pages long respectively. Now I know better than to equate length of a spec with value, but when you cut the boilerplate content out of these 14 and 9 pages, there is very little left for delivering on ambitious claims such as those Greg makes.

The reason is that these specs in no way provide interop between a system built using Liberty Alliance and a system built using WS-Federation. All they do is to allow each system to find out what spec the other uses.

One way to think about it is that we have an English speaker and a Korean speaker in the same room and they are not able to talk. What the two new specs do is put a lapel pin with a British flag on the english speakers and a lapel pin with a Korean flag on the korean speaker. Yes, this helps a bit. At least now the Korean speaker will know what the weird language that the other guy is speaking is and he can go to school and learn it. But just finding out what language the other guy speaks is a far cry from actually being able to communicate with him.

Even with these specs, a system based on Liberty Alliance and one based on WS-Federation are still incompatible and you cannot single sign-on from one to the other. Or rather, you can only if your client implements both. This is said explicitly in the Web SSO Interop Profile spec (look for the first line of page 5): “A compliant identity provider implementation MUST support both protocol suites”. Well, this isn’t interop, it’s duplication. Otherwise I could claim I have solved the problem of interoperability between English and Korean just by asking everyone to learn both languages. Not very convincing…

But of course Microsoft and Sun knew that they could get away with that in the press. For example, CNet wrote “The Web Single Sign-On Metadata Exchange Protocol and Web Single Sign-On Interoperability Profile will bridge Web identity management systems based on the Liberty and Web services specifications, the companies said”. As the Columbia Journalism Review keeps pointing out, real journalists don’t just report what people say, they check if it’s true. And in this case, it simply isn’t.

1 Comment

Filed under Business, Everything, Security, Standards, Tech

Modifying the state of a resource

Elements of state (aka properties) of a resource (such as WSDM properties, the properties of a CIM class or the attributes of an MBean) are a common component of management meta-models. They provide a way to expose information related to the resource. They are usually readable and sometimes also writeable. Reading the value usually means that the returned value is what the manageability representation believes reflects the state of the resource at the time the request is served. You generally can’t be sure that what the manageability representation sees is the actual state of the resource and that the state will not change one microsecond later, but within these usual restrictions, the semantics of a read are pretty clear. The semantics of a write are trickier as properties may represent an observed value, not one directly controllable (sometimes called a configuration value). For example, one cannot just “set” the GPS coordinate of a car and expect the car to be instantly teleported there. Does this mean that the GPS coordinate should not be writeable, or does it mean that setting them should be interpreted as an instruction to the car to do the best it can to drive itself to the location corresponding to the coordinates? Either is ok, as long as the contract is clear. Here are possible ways to handle this:

  • One approach is to make ALL properties (that is, the entire state of the resource) read-only and any change goes through operations. The state only changes as a side effect of operations, each of which has clear semantics. So instead of setting the GPS coordinates, one sends a DriveTo message containing the coordinates. The semantics are those of “DriveTo”, rather than reusing some generic semantics applicable to a class of “set” or “write” requests.
  • A variation on this is to explicitly select some elements of state as configuration elements that can be set by a generic “set” operation and leave aside non-configuration elements to be modified only as side-effects of operations. For example, threshold levels and ownership contact information can usually be written directly by the manageability implementation.
  • Another approach is to clearly acknowledge the difference between observed state and desired state and model all interactions as modifications to the desired state, leaving the manageability representation in charge of doing its best to reconcile the observed state with the desired state. In this case, a “set” or “write” operation is clearly a modification of the desired state, not necessarily the observed state.

1 Comment

Filed under Everything, Tech

Resource discovery with WSDM MUWS

Bryan has just published an article describing options for discovering resources using WSDM MUWS. A highly recommended read.

2 Comments

Filed under Everything, Standards, Tech

Services vs. Resources: the WSDM case

In an SOA, a service should not be tied to the resources that allow the service to be delivered. WSDM MUWS closely ties services with resources and in doing so it does not violate any SOA principle. I will show in this entry that these two sentences do not contradict each other.

Resources come and go and creating information systems that directly connect resources to one another results in brittle systems that don’t scale. Service-orientation, when well used, addresses this problem. Many have said this better than me before, like Jim: “Web Services are about hiding resources and exposing processes which operate on those resources”.

WSDM MUWS exposes a ResourceId property and manageability capabilities that are specifically tied to a given resource. But this “resource” is not the resource that makes it possible to deliver the MUWS service. It is the resource that the service has been created specifically to represent. Let’s illustrate this by contrasting two examples:

Service1 is a storage service. The “operational value” of this service is to store data. The right way to represent Service1 is in a way that is separated from the resource (in this case a storage array) that is used to provide the service. The service should expose its capabilities in terms of reading and writing data, not in terms of what SCSI disks are used. So that tomorrow I can replace the storage array with another one (or maybe with two smaller ones) and, assuming I replicate data correctly, the users of my service will not notice the change. A basic example of service-orientation. Now let’s look at Service2. Service2 is a management service (in MUWS terms, a “manageability endpoint”) used to manage the storage array from the previous example. The “operational value” of this service is not to store data in the array, it is to manage the array. And not any array, this specific array seating in my machine room. The resource used to provide the service is the Web services engine in which Service2 runs and whatever mechanism allows it to manage the storage array. In this case too, Service2 should be exposed in a way that is independent from the resource(s) that it relies on (like the Web service engine it runs on). But having it not be tied to the storage array makes would negate the very value this service provides, namely to manage a given storage array.

Of course in some cases it makes sense to embed the manageability endpoint inside the resource being managed in which case the resource being managed is also the resource that provides the service. But this is a corner case and in no way something requested by MUWS.

Separating the service from the resources that compose it is a good thing, but when the operational value of the service is exposed in terms of specific resources it is fine to explicitly attach the service to the resource. When deciding whether it is ok to let a resource show through a Web service, one needs to clearly understand whether it is a Service1 or Service2 type of situation.

Comments Off on Services vs. Resources: the WSDM case

Filed under Everything, Standards, Tech

First we think the Web is HTML; then HTTP; then we realize it’s URL.

As I was sitting in my car listening to KQED on the way back from work and I recently remembered an interview of Tim Berners-Lee by Terry Gross on Fresh Air that took place in 1999. TBL was promoting his book, Weaving the Web. At that time I was very familiar with Web technologies (first Web site in 1994 and I had been writing Web applications as CGIs more or less non-stop since 1995) but I hadn’t realized that the URL was the key building block of the Web, way ahead of HTTP and even more ahead of HTML. I don’t think I had ever asked myself the question, but if I had I would probably have sorted them backward. Hearing TBL in this interview describe how, before the Web, people would create small files that described where to find information in a human-readable way (I assume it must have been something like “telnet to this machine, use this logon/pwd, go to this directory, start this application, load this file”) really made me understand the importance of this URL thing I had taken for granted for many years. To this day I vividly remember this interview and the Eureka feeling when I realized the importance of URLs as an enabler for the Web.

I don’t know if the fact that this interview, which was targeted at the general audience of Fresh Air (more used to hearing Jazzmen interviewed than geeks), taught a Web-head like me something important is a testament to TBL’s vision, Terry Gross’ skills as an interviewer or my stupidity for not having grasped such a basic concept earlier.

Going back to WS-Addressing EPRs for a minute, what I was thinking recently is that these EPRs look a bit like the old “do this, do that” files that TBL talked about and that were replaced with URIs. Where “do this” becomes “put this header in your SOAP message”. Unlike the “do this” files, the instructions in the EPR can be machine-processed and that’s a key difference. But still, I can’t help getting this deja-vu feeling. Not that I have ever encountered these “do this” files myself but TBL made me see them one day in 1999.

[UPDATED 2011/9/27: Before pointing to this piece on Twitter (in response to this post by Joe Hewitt) I just have to change the awful title (for the record, the original title was “Thinking about EPRs like Proust”; yeah, I know). Whenever someone uses “la petite madeleine” (or Schrödinger’s cat for that matter) to illustrate a point you know it’s going to suck, so I removed it. And while I’m at it, I am replaced all the references to “URI” by “URL” which is less pedantic (and more accurate in this context). There. I usually don’t like to edit old entries, but this one was so bad it made almost no sense (not to mention the fact that no-one cares much about EPRs anymore).]

1 Comment

Filed under Everything, Standards, Tech

Reference properties are gone. Yawn…

The removal of reference properties (see WS-A issue #1) from the WS-A specification is a non-event. Today, most implementations and specs that use WS-Addressing are based on either the March 2003 or the March 2004 version, both of which have only one echoing mechanism, called “reference properties”. And, if they use it right, these specs and implementations don’t use the reference properties for identification, they just use the EPR as a pointer which happens to use an echoing mechanism (most of them don’t even care about the echoing mechanism and whether it is used). For these people, there is no change resulting from the removal of reference properties, other than just replacing the name “reference properties” with “reference parameters”.

The only place where there is a choice between two mechanisms (properties and parameters) is in the August 2004 version, the one submitted to W3C. Newer specs (like WSDM MUWS 1.0) reference this version because it has a few clarifications and because it feels more comfortable to use a reference residing on the W3C’s site (even though it doesn’t mean anything in the case of a member submission). But I am not aware of any implementation or spec that uses the August 2004 version and takes advantage of the difference between reference properties and reference parameters. For example, WS-Management makes use of both constructs but I don’t see how anything in WS-Management would stop working if all its reference properties were transformed into reference parameters and/or vice-versa (for example, why does wse:Identifier need to be a reference parameter rather than a reference property?). Once again, the removal of reference properties is not a change to WS-Addressing, it is just the end of a little escapade into the exciting world of inventing unnecessary semantics.

Comments Off on Reference properties are gone. Yawn…

Filed under Everything, Standards, Tech

@wsa:type=”parameter” instead of

Eager to prove wrong those who have a hard time picturing a January face to face meeting in Australia as little more than an excuse for some scuba diving, the WS-A working group is cranking through its issues list. One of the outcomes is that issue #8 is now resolved. The issue is about identifying which headers in a SOAP message are there because the EPR used to address the SOAP message required it. This problem is familiar to anyone who has ever been the victim of the common joke that consists in tricking someone who doesn’t speak a certain language into memorizing and later repeating a sentence in that language while misleading them on the meaning of the sentence. And waiting for them to embarrass themselves in public at the worst time.

The group closed this issue by introducing an attribute that can be added to these headers (wsa:type). Yes, the world really needed another type of “type”. BTW, the resolution will need to be tweaked as a result of another decision: issue #1 was resolved by getting rid of reference properties, leaving only reference parameters. More on this in a future post, but at least this gives an opportunity to replace wsa:type=”parameter” with wsa:parameter=”true” and avoid yet-another-type.

Going back to issue #8, I am glad the group somewhat acknowledged the problem but this doesn’t solve it. For two reasons:

  1. The schema of the header element might not allow this attribute to be added
  2. Even if the attribute is present, the recipient of the SOAP message can pretend to not understand it (“I don’t care about this attribute and all the other WS-A crap, all I know is that you sent me the ‘I owe you $1,000’ header and you signed it so now pay up!”)

    The way to solve both of these problems was to create one additional SOAP header (in the WS-A namespace) that could be marked mustUnderstand=”true” and points to all the headers in the message that are there because “the EPR told me so”. I proposed this (the wsa:CoverMyRearside approach) in the one and only message I have sent to the WS-A WG, but obviously I wasn’t very convincing. Since I don’t participate in the WG I might never understand what is wrong with this approach. What really surprises me is that it wasn’t even considered. The issues list shows only 3 proposals but the minutes of the face to face show that there was a 4th option considered, which comes a bit closer. Basically, it is the same as the adopted solution with the addition of a WS-A-defined header (that could be marked mustUnderstand=”true”) which states that “by processing this header you agree that you understand the wsa:type attribute when present in other headers”. This is not very elegant in my mind and doesn’t solve (1) but it does solve (2).

    Interestingly, even though the wsa:CoverMyRearside header approach was not chosen by the WG, nothing stops me from using this approach: I can defined and use a header in my namespace that I’ll mark mustUnderstand=”true” and that will point to the headers that are there only because “the EPR told me so”. The problem of course is that my namespace is not going to be nearly as well-known as the WS-A namespace and most people will fail on processing my messages because it contains a mustUnderstand=”true” header they do not understand. So in practice I can’t do this. That being said, I wouldn’t be surprised if some spec somewhere one day decided that the mechanism in WS-Addressing is not good enough and that they should take on the task to define such a header because they need this protection.

    Comments Off on @wsa:type=”parameter” instead of

    Filed under Everything, Security, Standards, Tech

    From UPS to EPR

    A packaged was shipped to me through UPS. As usual, I received an email message informing me that it had shipped and giving me a URI to track its progress. This is what the URI looked like (after changing the tracking number and inserting a few line breaks):

    http://wwwapps.ups.com/WebTracking
    /processRequest?HTMLVersion=5.0&Requester=NES
    &AgreeToTermsAndConditions=yes&loc=en_US
    &tracknum=123123123123

    The interesting thing to notice, is that there is a parameter in the URI, called “AgreeToTermsAndConditions” and its value is set to “yes”. If you do a GET on this URI, you will receive, as expected, the description of the status of the shipment. On the other hand, if you go to the UPS Web site armed with just the tracking number you have to check a box that reads “By selecting this box and the ‘Track’ button, I agree to these Terms and Conditions” before you can track the shipment. It seems pretty clear that the “AgreeToTermsAndConditions” parameter is in the URI in order to plug into the same Web application from the email link as from the Web page and this application was designed to check that the parameter is present and set to “yes”.

    This has several similarities with some of the discussions around WS-Addressing. First, it illustrates the need to plug into an application in a place where the application might not have been designed to allow you to do so. In this case, we can imagine that the tracking application was designed with the assumption that people would only invoke it from the UPS web site. One day UPS decided it would be nice to send in email a link that takes people directly to the status of a particular shipment rather than just tell them to “go to ups.com and enter your tracking ID in the tracking form”. One important reason for pushing back on the idea of wrapping reference properties is that it would prevent such a scenario in a SOAP world. For this reason I agree that a wrapper for reference properties is a bad idea and if reference properties remain in WS-Addressing the way to fix this mechanism is to leave them as SOAP headers but add a WS-Addressing-defined SOAP header to list headers that were added to a message only because an EPR requires it, with no implication that the sender stands behind any semantic that might be attached to them.

    When I write about “fixing” reference properties in the previous sentence, I am referring to the fact that the current version of WS-Addressing creates a lot of confusion as to the implications of sending SOAP headers and whether I can be held liable by anything that is in a SOAP header I send (and potentially sign). This is the second thing that this UPS URI illustrates by analogy. As a human I get a sense of what a parameter called “AgreeToTermsAndConditions” corresponds to (even though the URI doesn’t tell me what these terms and conditions are). But what if the parameter name was shortened to its acronym “ATTAC”? In any case, I am not expected to understand the composition of this URI, I should be able to treat it as opaque. Just like resource properties. And for this reason, when I do a GET on the URI I am not bound by whatever the presence of this parameter in the URI is assumed to mean by UPS. This means that I can NEVER be bound by the content of a URI I dereference because how can one prove that I understand the semantic of the URI. Even when I fill a form on a Web site, I don’t know (unless I check the HTML) how the resulting URI (assuming the form uses GET) will come out. There might well be a hidden parameter in the form.

    In a SOAP world, this can be fixed by meaningful, agreed-upon, headers. If people agree that by sending (and signing) a SOAP header you are making a statement, then you can build systems that can rely on the ability to make such statements as “I understand and agree to the set of terms and conditions identified by a given QName”. But this breaks down if people are able to say “I didn’t understand I was saying this, I was just echoing what you told me to say”. This is what the current WS-Addressing spec does, and what needs to be fixed. Let’s see how the UPS URI could translate to an endpoint reference. For this, we need to consider two scenarios, both of them equally valid.

    Scenario 1: legacy integration

    In this scenario, the UPS people decide they do not require the invoker to actually make a statement about the terms and conditions. They just need a SOAP header called “I agree to the terms and conditions” to be set to true because this is how their application is currently programmed (it will fail if it doesn’t see it). In this case, it is perfectly reasonable to put a “I agree to terms and conditions” element as a reference property and this element will be sent as a SOAP header, preventing the application from failing. But in order for SOAP headers to be used to make a statement in some cases, there needs to be a way to expressed when, as in this scenario, the invoker is not making a statement by including it. Thus my earlier proposal of a wsa:CoverMyRearside header (that points to all the headers I include for no reason other than because an EPR asks me to). The other option, as written down by Dims is to add an attribute in each such header. But there are two main drawbacks to this approach: (1) unlike a new SOAP header, I can’t mark an attribute “mustUnderstand=true” (Dims’ initial proposal actually had a SOAP header with “mustUnderstand=true” for this very reason) and (2) some elements might not allow arbitrary attributes to be added at the top level.

    Scenario 2: meaningful header

    In this second scenario, the UPS people want to make sure, before they return the tracking information, that the invoker has made a statement that it understands and agrees to the terms and conditions. In this case, it makes no sense to put a “”I agree to the terms and conditions” element as a reference property as what is intended is not for such an element to be echoed blindly but to be used to make an explicit statement. In this scenario, the EPR sent by UPS would contain all the opaque elements, those sent for the benefit of the UPS application but by which the invoker is not making a statement (from the look of the URI, this would be “HTMLVersion”, “Requester”, “loc” and “tracknum”). But the “agree to terms and conditions” header would not be specified as a reference property, it would be listed in the WSDL description of the service. And when the invoker sends this header, it would not be included in the wsa:CoverMyRearside header because the invoker is indeed making a statement by sending it.

    Comments Off on From UPS to EPR

    Filed under Everything, Security, Standards, Tech

    WSRF/WSN overview

    For those who would like a short introduction to WSRF and WSN, there is a new one on HP’s DRC site based on presentation I gave a few months ago. Thanks Chris and Kathleen for putting this up.

    Comments Off on WSRF/WSN overview

    Filed under Everything, Standards, Tech

    That’s not an identifier. THIS is an identifier (say it with a Crocodile Dundee accent)

    I already explained that I agree with Paco’s view that EPRs are not identifiers. Pankaj also provided a concrete example of why confusing references and identifiers causes problems. Paco just sent a new, better explanation of his earlier point, in the format of a formal proposal to the WS-Addressing WG. His proposal and its justification are a must read. He starts with a “what is required from an identifier” paragraph, which reads:

    An identifier to be useful must allow meaningful comparison for identity or “sameness”. This requires them to overall unique and unambiguous, otherwise no meaningful comparison is possible. Moreover one could argue that to be really useful identifiers should not be reused once they’ve been made invalid.

    Compare this with the specification for the ResourceId property defined in MUWS Part 1 (note: this is a link to the current working draft as a Word document, not yet a committee draft). Some highlights that match very well with Paco’s expectations for a *real* identifier:

    • Globally unique: A manageability endpoint MUST create the ResourceId URI in a way that ensures that the ResourceId is unique to the resource managed through the manageability endpoint and globally unique.
    • Uniqueness in time: A ResourceId MUST NOT be reused by the implementation of a manageability endpoint for another resource, even after the original resource no longer exists.
    • Consistency across endpoints: An implementation of a manageability endpoint SHOULD use a ResourceId that is suggested by the characteristics of a resource.

    And the spec goes on to define in more details why/how implementers should ensure that difference manageability endpoints for the same resource return the same ResourceId, persistence of the ResourceId in time and how to establish “sameness” when for some reason different manageability endpoints for the same resource are unable to return the same ResourceId (correlatable properties). Go ahead and read it or wait a couple of weeks if you want to see a committee draft as a clean PDF rather than a Word document with change tracking turned on.

    Comments Off on That’s not an identifier. THIS is an identifier (say it with a Crocodile Dundee accent)

    Filed under Everything, Standards, Tech

    Whose policy is it anyway?

    Greg recently posted an interesting article on what’s ahead of us in terms of really using policies in conjunction with Web services. I agree with him that the interesting question is not in “features and properties” versus WS-Policy. Whether you use one, the other, or a combination of the two you’ve still barely scratched the surface. I also agree that policies should not be tied with WSDL descriptions (a problem of F&P). But a key thing I would add to Greg’s list of what people need to be able to do with policies is define more precisely what “components” of the architecture policies are attached to. Yes, the “service” component is not sufficient as policies can be attached to more granular or less granular levels. But what are these levels? Who defines them? How do we come to agreement on them?

    WSDM MUWS 1.0 Part 1 (coming in December) will define such basic components as “property”, “operation” and “event” that can have policies attached to them. These are components that are more granular than a “service” or an “endpoint”. Then you have relationships (defined in MUWS Part 2) that can also have attached policies. And you can have policies applied at a higher granularity level than the service, for example at the level of a business process made up of several services. Who will define these components?

    Once we have these policy-capable components defined, we can go back to Greg’s question about attaching policies to them. “Multiple methods of association”, as he suggests, is one possible approach. Another way is to make more daring (shall we say) usage of existing addressing methods. For example the way WS-Management does it in its section 3. But this comes at the cost of throwing down the drain the benefits associated with opaque reference properties in WS-Addressing. Is it worth it? Any other alternative?

    Comments Off on Whose policy is it anyway?

    Filed under Everything, Standards, Tech