William Vambenepe's blog

WSRF/CDL/WS-Addressing complexity used as a security barrier? Ouch!

The question recently came back on the WS-Addressing mailing list of whether Reference Parameters can/should be used as the SOAP equivalent of cookies. Something more along the lines of session management than addressing. See Peter Hendry’s email for a clear description of his use case. The use case is reasonable but I don’t think this is what WS-Addressing is really for as I explain in bullet #3 of this post. What interested me more was the response that came from Conor Cahill and his statement that AOL is implementing an “EndpointReferenceUpdate” element that can be returned in the response to tell the sender to update the EPR. I am not fond of this as a mechanism for session management, but I can see one important benefit of this mechanism: getting hold of a “good” EPR for more efficient addressing. Here is an example application:

Imagine a Web services that represents the management interface of a business process engine. That Web service provides access to all the currently running business process instances in the engine (think Service Group if you’re into WSRF). Imagine that this Web service supports a SOAP header called “target” and that header is defined to contain an XPath statement. When receiving a message containing a “target” header, the Web service will look for the (for the sake of simplicity let’s assume there can only be one) business process instance for which this XPath statement returns “true” when evaluated on the XML representation of the state of the business process instance. And the Web service will then interpret the message to be targeted at that business process instance. This is somewhat similar to WS-Management’s “SelectorSet”. A sender can use this mechanism to address a specific business process instance based on the characteristics of that instance (side note: whether the sender understands and builds this header itself or whether it gets it as a Reference Parameter from an EPR is orthogonal). But this can be a very expensive dispatching mechanism. The basic implementation would require the Web service to evaluate an XPath statement on each and every business process instance state document. Far from optimal. This is where Conor’s “EndpointReferenceUpdate” can come in handy. After doing once the XPath evaluation work of finding out which business process instance the sender wants to address, the Web service can return a more optimized EPR to be used to address that instance, one that is a lot easier to dispatch on. This kind of scenario is a lot more relevant in my perspective to the work of the WS-Addressing working group than the session example.

An important consequence of a mechanism such as “EndpointReferenceUpdate” is that it makes it critical that the Web service be able to tell which SOAP headers are in the message as a result of being in the EPR used by the sender and which ones were added by the sender on purpose. For example, if a SOAP message comes in with headers “a”, “b” and “c” and the Web service assumes that “a” and “b” were in the EPR and “c” was added by the invoker, then the new EPR returned as part of “EndpointReferenceUpdate” will only be a replacement for “a” and “b” and the Web service will still expect “c” to be added by the sender. But if in fact “c” also came from a reference parameter in the EPR used by the sender then follow-up messages will be incomplete. This puts more stress and responsibilities on the already weak @isReferenceParameter attribute. And, by encouraging people to accept EPRs from more and more sources, it puts EPR consumers are even greater risk for the problems described in bullet (1) of this objection.

EPR (Endpoint References, from WS-Addressing) are a shiny and exciting toy. But a sharp one too. So here is my contribution to try to prevent fingers from being cut and eyes from being poked out.

So far I have seen EPRs used for five main reasons, not all of them very inspired:

1) “Dispatching on URIs is not cool”

Some tools make it hard to dispatch on URI. As a result, when you have many instances of the same service, it is easier to write the service if the instance ID is in the message rather than in the endpoint URI. Fix the tools? Nah, let’s modify the messages instead. I guess that’s what happens when tool vendors drive the standards, you see specifications that fit the tools rather than the contrary. So EPRs are used to put information that should be in the URI in headers instead. REST-heads see this as a capital crime. I am not convinced it is so harmful in practice, but it is definitely not a satisfying justification for EPRs.

2) “I don’t want to send a WSDL doc around for just the endpoint URI”

People seem to have this notion that the WSDL is a “big and static” document and the EPR is a “small and dynamic” document. But WSDL was designed to allow design-time and run-time elements to be separated if needed. If all you want to send around is the URI at which the service is available, you can just send the URI. Or, if you want it wrapped, why not send a soap:address element (assuming the binding is well-known). After all, in many cases EPRs don’t contain the optional service element and its port attribute. If the binding is not known and you want to specify it, send a around a wsdl:port element which contains the soap:address as well as the QName of the binding. And if you want to be able to include several ports (for example to offer multiple transports) or use the wsdl:import mechanism to point to the binding and portType, then ship around a simplified wsdl:descriptions with only one service that itself contains the port(s) (if I remember correctly, WS-MessageDelivery tried to formalize this approach by calling a WSRef a wsdl:service element where all the ports use the same portType). And you can hang metadata off of a service element just as well as off of an EPR.

For some reason people are happy sending an EPR that contains only the address of the endpoint but not comfortable with sending a piece of WSDL of the same size that says the same thing. Again, not a huge deal now that people seem to have settled on using EPRs rather than service elements, but clearly not a satisfying justification for inventing EPRs in the first place.

3) “I can manage contexts without thinking about it”

Dynamically generated EPRs can be used as a replacement for an explicit context mechanism, such as those provided by WS-Context and WS-Coordination. By using EPRs for this, you save yourself the expense of supporting yet-another-spec. What do you loose? This paper gives you a detailed answer (it focuses on comparing EPRs to WS-Context rather than WS-Coordination for pretty obvious reasons, but I assume that on a purely technical level the authors would also recommend WS-Coordination over EPRs, right Greg?). In a shorter and simplified way, my take on the reason why you want to be careful using dynamic EPRs for context is that by doing so you merge the context identifier on the one hand and the endpoint with which you use this context on the other hand into one entity. Once this is done you can’t reliably separate them and you loose potentially valuable information. For example, assume that your company buys from a bunch of suppliers and for each purchase you get an EPR that allows you to track the purchase as it is shipped. These EPRs are essentially one blob to you and the only way to know which one comes through FedEx versus UPS is to look at the address and try to guess based on the domain name. But you are at the mercy of any kind of redirection or load-balancing or other infrastructure reason that might modify the address. That’s not a problem if all you care about is checking the ETA on the shipment, each EPR gives you enough information to do that. But if you also want to consolidate the orders that UPS is delivering to you or if you read in the paper about a potential UPS drivers strike and want to see how it would impact you, it would be nice to have each shipment be an explicit context Id associated to a real service (UPS or FedEx), rather than a mix of both at the same time. This way you can also go to UPS.com, ask about your shipments and easily map each entry returned to an existing shipment you are tracking. With EPRs rather than explicit context you can’t do this without additional agreements.

The ironic thing is that the kind of mess one can get into by using dynamic EPRs too widely instead of explicit context is very similar in nature to the management problems HP OpenView software solves. Discovery of resources, building relationship trees, impact analysis, event correlation, etc. We do it by using both nicely-designed protocols/models (the clean way) and by using heuristics and other hacks when needed. We do what it takes to make sense of the customer’s system. So we could just as well help you manage your shipments even if they were modeled as EPRs (in this example). But we’d rather work on solving existing problems and open new possibilities than fix problems that can be avoided. And BTW using dynamic EPRs is not always bad. Explicit contexts are sometimes overkill. But keep in mind that you are loosing data by bundling the context with the endpoint. Actually, more than loosing data, you are loosing structure in your data. And these days the gold is less in the raw data than in its structure and the understanding you have of it.

4) “I use reference parameters to create new protocols, isn’t that cool!”

No it’s not. If you want to define a SOAP header, go ahead: define an XML element and then describe the semantics associated with this element when it appears as a SOAP header. But why oh why define it as a “reference parameter” (or “reference property” depending on your version of WS-A)? The whole point of an EPR is to be passed around. If you are going to build the SOAP message locally, you don’t need to first build an EPR and then deconstruct it to extract the reference parameters out of it and insert them as SOAP headers. Just build the SOAP message by putting in the SOAP headers you know are needed. If your tooling requires going through an EPR to build the SOAP message, fine, that’s your problem, but don’t force this view on people who may want to use your protocol. For example, one can argue for or against the value of WS-Management’s System and SelectorSet as SOAP headers, but it doesn’t make sense to define those as reference parameters rather than just SOAP headers (readers of this blog already know that I am the editor of the WSDM MUWS OASIS standard with which WS-Management overlaps so go ahead and question my motives for picking on WS-Management). Once they are defined as SOAP headers, one can make the adventurous decision to hard-code them in EPRs and to send the EPRs to someone else. But that’s a completely orthogonal decision (and the topic of the fifth way EPRs are used – see below). But using EPRs to define protocols is definitely not a justification for EPRs and one would have a strong case to argue that it violates the opacity of reference parameters specified in WS-Addressing.

5) “Look what I can do by hard-coding headers!”

The whole point of reference parameters is to make people include elements that they don’t understand in their SOAP headers (I don’t buy the multi-protocol aspect of WS-Addressing, as far as I am concerned it’s a SOAP thing). This mechanism is designed to open a door to hacking. Both in the good sense of the term (hacking as a clever use of technology, such as displaying Craig’s list rental data on top of Google maps without Craig’s List or Google having to know about it), and in the bad sense of the term (getting things to happen that you should not be able to make happen). Here is an example of good use for reference parameters: if the Google search SOAP input message accepted a header that specifies what site to limit the search on (equivalent to adding “site:vambenepe.com” in the Google text box on Google.com), I could distribute to people an EPR to the vambenepe.com search service by just giving them an EPR pointing to the Google search service and adding a reference parameter that corresponds to the header instructing Google to limit the search to vambenepe.com.

Some believe this is inherently evil and should be stopped, as expressed in this formal objection. I think this is a useful mechanism (to be used rarely and carefully) and I would like to see it survive. But there are two risks associated with this mechanism that people need to understand.

The first risk is that EPRs allow people to trick others into making statements that they don’t know they are making. This is explained in the formal objection from Anish and friends as their problem #1 (“Safety and Security”) and I agree with their description. But I don’t agree with the proposed solutions as they prevent reference parameters to be treated by the service like any other SOAP header. Back last November I made an alternative proposal, using a wsa:CoverMyRearside element that would not have this drawback and I know other people have made similar proposals. In any case, this risk can and should be addressed by the working group before the specification becomes a Recommendation or people will stop accepting to process reference parameters after a few high-profile hacks. Reference parameters will become the ActiveX of SOAP.

The second risk is more subtle and that one cannot be addressed by the specification. It is the fragility that will result from applications that share too many assumptions. I get suspicious when someone gives me directions to their house with instructions such as “turn left after the blue van” or “turn right after the barking dog”, don’t you? “We’re the house after the green barn” is a little better but what if I want to re-use these directions a few years later. What’s the chance that the barn will be replaced or repainted? EPRs that contain reference parameters pose the same problem. Once you’ve sent the EPR, you don’t know how long it will be around, you don’t know who it will get forwarded to, you don’t know what the consumer will know. You need to spend at least as much efforts picking what data you use as a reference parameter (if anything) as you spend designing schemas and WSDL documents. If your organization is smart enough to have a process to validate schemas (and you need that), that same process should approve any element that is put in a reference parameter.

Or you’ll poke your eye out.

This morning I learned that Microsoft and Sun had a public event where the CEOs reported on a year of working together. This is a follow-up to Greg Papadopoulos’ report on the progress of the “technical collaboration”. In that post, Greg told us about the amazing technical outcomes of the work between the two companies and, being very familiar with the specs he was referring to, I couldn’t help but point out that the result of the “technical collaboration” he was talking about looked a lot like Sun rubber-stamping a bunch of Microsoft specifications without much input from Sun engineers.

So when I heard this morning that the two companies were coming out publicly with the result of their work, I thought it would be fair for me to update my blog and include this information.

Plus, reading the press release and Greg’s Q&A session, it sounded pretty impressive and it would have been bad faith from my part to not acknowledge that indeed Greg actually had something to brag about, it just wasn’t yet public at the time. In effect, it sounded like they had found a way to make the Liberty Alliance specs and WS-Federation interoperate with one another.

From Greg’s Q&A: “In a nutshell, we resolved and aligned what Microsoft was trying to accomplish with Passport and the WS-Federation with what we’ve been doing with the Liberty Alliance. So, we’ve agreed upon a way to enable single sign-on to the Internet (whether through a .NET service or a Java Enterprise System service), and federate across those platforms based on service-level agreements and/or identity agreements between those services. That’s a major milestone.”

Yes Greg, it would have been. Except this is not what is delivered. The two specs that are supposed to support these claims are Web SSO MEX and Web SSO Interop Profile. Which are 14 and 9 pages long respectively. Now I know better than to equate length of a spec with value, but when you cut the boilerplate content out of these 14 and 9 pages, there is very little left for delivering on ambitious claims such as those Greg makes.

The reason is that these specs in no way provide interop between a system built using Liberty Alliance and a system built using WS-Federation. All they do is to allow each system to find out what spec the other uses.

One way to think about it is that we have an English speaker and a Korean speaker in the same room and they are not able to talk. What the two new specs do is put a lapel pin with a British flag on the english speakers and a lapel pin with a Korean flag on the korean speaker. Yes, this helps a bit. At least now the Korean speaker will know what the weird language that the other guy is speaking is and he can go to school and learn it. But just finding out what language the other guy speaks is a far cry from actually being able to communicate with him.

Even with these specs, a system based on Liberty Alliance and one based on WS-Federation are still incompatible and you cannot single sign-on from one to the other. Or rather, you can only if your client implements both. This is said explicitly in the Web SSO Interop Profile spec (look for the first line of page 5): “A compliant identity provider implementation MUST support both protocol suites”. Well, this isn’t interop, it’s duplication. Otherwise I could claim I have solved the problem of interoperability between English and Korean just by asking everyone to learn both languages. Not very convincing…

But of course Microsoft and Sun knew that they could get away with that in the press. For example, CNet wrote “The Web Single Sign-On Metadata Exchange Protocol and Web Single Sign-On Interoperability Profile will bridge Web identity management systems based on the Liberty and Web services specifications, the companies said”. As the Columbia Journalism Review keeps pointing out, real journalists don’t just report what people say, they check if it’s true. And in this case, it simply isn’t.

There is a new WSDL validation tool on IBM’s AlphaWorks site: the Web Services Interface Definition for Intrusion Defense. This is an Eclipse plug-in that checks out your WSDL and flags “any interface feature that could open a door to hacker attacks”. What does it mean in practice? Well, it flags any usage of xsd:any (or xsd:anyType or xsd:anySimpleType) anywhere in your schemas. It also complains if you have elements with maxOccurs=”unbounded”. And more of the same. The result is that this excludes pretty much any existing schema definition. And most of the useful ones one can think of.

The payload of XML messages should reflect the business logic of the service and not the convenience of the implementer. Go tell the line of business manager that the “checkout” operation should be modified so that the number of items in the shopping cart has a hard-coded limit. Go tell the print shop that they can’t accept XHTML documents as input. It is the implementer’s job (and by that I include the runtime, IDE and tools) to make sure the message processing code (be it at the plumbing level or the business level) doesn’t expose security holes.

Eager to prove wrong those who have a hard time picturing a January face to face meeting in Australia as little more than an excuse for some scuba diving, the WS-A working group is cranking through its issues list. One of the outcomes is that issue #8 is now resolved. The issue is about identifying which headers in a SOAP message are there because the EPR used to address the SOAP message required it. This problem is familiar to anyone who has ever been the victim of the common joke that consists in tricking someone who doesn’t speak a certain language into memorizing and later repeating a sentence in that language while misleading them on the meaning of the sentence. And waiting for them to embarrass themselves in public at the worst time.

The group closed this issue by introducing an attribute that can be added to these headers (wsa:type). Yes, the world really needed another type of “type”. BTW, the resolution will need to be tweaked as a result of another decision: issue #1 was resolved by getting rid of reference properties, leaving only reference parameters. More on this in a future post, but at least this gives an opportunity to replace wsa:type=”parameter” with wsa:parameter=”true” and avoid yet-another-type.

Going back to issue #8, I am glad the group somewhat acknowledged the problem but this doesn’t solve it. For two reasons:

1. The schema of the header element might not allow this attribute to be added
2. Even if the attribute is present, the recipient of the SOAP message can pretend to not understand it (“I don’t care about this attribute and all the other WS-A crap, all I know is that you sent me the ‘I owe you $1,000′ header and you signed it so now pay up!”)

The way to solve both of these problems was to create one additional SOAP header (in the WS-A namespace) that could be marked mustUnderstand=”true” and points to all the headers in the message that are there because “the EPR told me so”. I proposed this (the wsa:CoverMyRearside approach) in the one and only message I have sent to the WS-A WG, but obviously I wasn’t very convincing. Since I don’t participate in the WG I might never understand what is wrong with this approach. What really surprises me is that it wasn’t even considered. The issues list shows only 3 proposals but the minutes of the face to face show that there was a 4th option considered, which comes a bit closer. Basically, it is the same as the adopted solution with the addition of a WS-A-defined header (that could be marked mustUnderstand=”true”) which states that “by processing this header you agree that you understand the wsa:type attribute when present in other headers”. This is not very elegant in my mind and doesn’t solve (1) but it does solve (2).

Interestingly, even though the wsa:CoverMyRearside header approach was not chosen by the WG, nothing stops me from using this approach: I can defined and use a header in my namespace that I’ll mark mustUnderstand=”true” and that will point to the headers that are there only because “the EPR told me so”. The problem of course is that my namespace is not going to be nearly as well-known as the WS-A namespace and most people will fail on processing my messages because it contains a mustUnderstand=”true” header they do not understand. So in practice I can’t do this. That being said, I wouldn’t be surprised if some spec somewhere one day decided that the mechanism in WS-Addressing is not good enough and that they should take on the task to define such a header because they need this protection.

A packaged was shipped to me through UPS. As usual, I received an email message informing me that it had shipped and giving me a URI to track its progress. This is what the URI looked like (after changing the tracking number and inserting a few line breaks):

http://wwwapps.ups.com/WebTracking

/processRequest?HTMLVersion=5.0&Requester=NES
&AgreeToTermsAndConditions=yes&loc=en_US
&tracknum=123123123123

The interesting thing to notice, is that there is a parameter in the URI, called “AgreeToTermsAndConditions” and its value is set to “yes”. If you do a GET on this URI, you will receive, as expected, the description of the status of the shipment. On the other hand, if you go to the UPS Web site armed with just the tracking number you have to check a box that reads “By selecting this box and the ‘Track’ button, I agree to these Terms and Conditions” before you can track the shipment. It seems pretty clear that the “AgreeToTermsAndConditions” parameter is in the URI in order to plug into the same Web application from the email link as from the Web page and this application was designed to check that the parameter is present and set to “yes”.

This has several similarities with some of the discussions around WS-Addressing. First, it illustrates the need to plug into an application in a place where the application might not have been designed to allow you to do so. In this case, we can imagine that the tracking application was designed with the assumption that people would only invoke it from the UPS web site. One day UPS decided it would be nice to send in email a link that takes people directly to the status of a particular shipment rather than just tell them to “go to ups.com and enter your tracking ID in the tracking form”. One important reason for pushing back on the idea of wrapping reference properties is that it would prevent such a scenario in a SOAP world. For this reason I agree that a wrapper for reference properties is a bad idea and if reference properties remain in WS-Addressing the way to fix this mechanism is to leave them as SOAP headers but add a WS-Addressing-defined SOAP header to list headers that were added to a message only because an EPR requires it, with no implication that the sender stands behind any semantic that might be attached to them.

When I write about “fixing” reference properties in the previous sentence, I am referring to the fact that the current version of WS-Addressing creates a lot of confusion as to the implications of sending SOAP headers and whether I can be held liable by anything that is in a SOAP header I send (and potentially sign). This is the second thing that this UPS URI illustrates by analogy. As a human I get a sense of what a parameter called “AgreeToTermsAndConditions” corresponds to (even though the URI doesn’t tell me what these terms and conditions are). But what if the parameter name was shortened to its acronym “ATTAC”? In any case, I am not expected to understand the composition of this URI, I should be able to treat it as opaque. Just like resource properties. And for this reason, when I do a GET on the URI I am not bound by whatever the presence of this parameter in the URI is assumed to mean by UPS. This means that I can NEVER be bound by the content of a URI I dereference because how can one prove that I understand the semantic of the URI. Even when I fill a form on a Web site, I don’t know (unless I check the HTML) how the resulting URI (assuming the form uses GET) will come out. There might well be a hidden parameter in the form.

In a SOAP world, this can be fixed by meaningful, agreed-upon, headers. If people agree that by sending (and signing) a SOAP header you are making a statement, then you can build systems that can rely on the ability to make such statements as “I understand and agree to the set of terms and conditions identified by a given QName”. But this breaks down if people are able to say “I didn’t understand I was saying this, I was just echoing what you told me to say”. This is what the current WS-Addressing spec does, and what needs to be fixed. Let’s see how the UPS URI could translate to an endpoint reference. For this, we need to consider two scenarios, both of them equally valid.

Scenario 1: legacy integration

In this scenario, the UPS people decide they do not require the invoker to actually make a statement about the terms and conditions. They just need a SOAP header called “I agree to the terms and conditions” to be set to true because this is how their application is currently programmed (it will fail if it doesn’t see it). In this case, it is perfectly reasonable to put a “I agree to terms and conditions” element as a reference property and this element will be sent as a SOAP header, preventing the application from failing. But in order for SOAP headers to be used to make a statement in some cases, there needs to be a way to expressed when, as in this scenario, the invoker is not making a statement by including it. Thus my earlier proposal of a wsa:CoverMyRearside header (that points to all the headers I include for no reason other than because an EPR asks me to). The other option, as written down by Dims is to add an attribute in each such header. But there are two main drawbacks to this approach: (1) unlike a new SOAP header, I can’t mark an attribute “mustUnderstand=true” (Dims’ initial proposal actually had a SOAP header with “mustUnderstand=true” for this very reason) and (2) some elements might not allow arbitrary attributes to be added at the top level.

Scenario 2: meaningful header

In this second scenario, the UPS people want to make sure, before they return the tracking information, that the invoker has made a statement that it understands and agrees to the terms and conditions. In this case, it makes no sense to put a “”I agree to the terms and conditions” element as a reference property as what is intended is not for such an element to be echoed blindly but to be used to make an explicit statement. In this scenario, the EPR sent by UPS would contain all the opaque elements, those sent for the benefit of the UPS application but by which the invoker is not making a statement (from the look of the URI, this would be “HTMLVersion”, “Requester”, “loc” and “tracknum”). But the “agree to terms and conditions” header would not be specified as a reference property, it would be listed in the WSDL description of the service. And when the invoker sends this header, it would not be included in the wsa:CoverMyRearside header because the invoker is indeed making a statement by sending it.

I am a bit confused about the attempt of the credit report companies to “protect the link” to the new site where one can get free credit reports (see the bottom of this article). I understand the problem of phishing and I agree that this site will be a very tempting target for phishing. The “solution” the people at annualcreditreport.com came up with is that links to their site won’t work unless the link is either on the FTC web site or one of the three credit report companies. For example, try clicking here and it should give you an error. This is presumably done by checking the “Referer” HTTP header. Yes, it’s “referer”, not “referrer”, an example of how authors of Web browsers are normatively required to make spelling mistakes, this should keep all of us spec editors on our toes (which, as a side note, happens to be literally the case for me this week because I slammed my heel on the edge of the swimming pool by doing an aggressive flip turn a bit too close to the wall during a Master’s swimming training session).

Anyway, back to the “protect the link” story. Let’s see how this would work. I have in front of me a hypertext link to their site. If I know that the site has this “protection” then why would I click on it? If it works then it’s a fake and I don’t want to use it and if it’s not a fake then it won’t work. In any case I know I’ll have to enter the URL by hand so I’ll do it right away. And if I am not aware of this behavior then I will click on the link. If it works (because the link is a fake), then I have no reason to suspect anything fishy and I’m in trouble. If it doesn’t work (because the link is real) then I’ll see that I have to enter the URL by hand and I will be out of trouble but I wasn’t in trouble to start with since the link was correct. In either of these four permutations this system doesn’t do any good.

The scary thing is that when people learn about this system, they might actually be more trustful with sites that implement something like this. But what stops a phishing site from doing the exact same thing, giving people an error message unless they type the URL by hand? Nothing. This reminds me of an email I once got from my bank to “educate” me about phishing. With “helpful” advice such as “we will never request personal information from you until after you are logged into our site using your username and password”. Hello? I give them my credentials and this supposedly authenticates them to me?