William Vambenepe's blog

If you care about exposing or accessing MBeans via WS-Management, now is a good time to read the public review draft of the JSR262 spec.

JSR262 is very much on the “manageability” side of the “manageability vs. management integration” chasm, which is not the most exciting side to me. But more commonality in manageability protocols is good, I guess, and this falls inside the WS-Management window of opportunity so it may help tip the balance.

There is also a nice white paper which does a nice job of retracing the history from JMX to JMX Remote API to JSR 262 and the different efforts along the way to provide access to the JMX API from outside of the local JVM. The white paper is actually too accurate for its own good: it explains well that models and protocols should be orthogonal (there is a section titled “The Holy Grail of Management: Model, Data and Protocol Independence”) which only highlights the shortcomings of JSR262 in that regard.

In a what looks from the outside like a wonderful exercise of “when you have a hammer” (and also “when you work in a hammer factory” like the JCP), this whole Java app management effort has been API-driven rather than model-driven. What we don’t get out of all this is a clearly defined metamodel and a set of model elements for Java apps with an XML serialization that can be queried and updated. What we do get is a mapping of “WS-Management protocol operations to MBean and MBean server operations” that “exposes JMX technology MBeans as WS-Management resources”.

Yes it now goes over HTTP so it can more easily fool firewalls, but I am yet to see such a need in manageability scenarios (other than from hackers who I am sure are very encouraged by the development). Yes it is easier for a non-Java endpoint to interact with a JSR262 endpoint than before but this is an incremental improvement above the previous JMX over RMI over IIOP because the messages involved still reflect the underlying API.

Maybe that’s all ok. There may very well not be much management integration possible at the level of details provided by JMX APIs. Management integration is probably better served at the SCA and OSGi levels anyway. Having JSR262 just provide incremental progress towards easier Java manageability by HP OVO and the like may be all we should ask of it. I told some of the JSR262 guys, back when they were creating their own XML over HTTP protocol to skirt the WS-Management vs. WSDM debate, that they should build on WS-Management and I am glad they took that route (no idea how much influence my opinion had on this). I just can’t get really excited about the whole thing.

All the details on the current status of JSR262 on Jean-Francois Denise’s blog.

I just noticed this result from the 2007 DMTF member survey (taken a year ago, but as far as I can tell just released now). When asked what their “most important interoperability priority” is, members made it pretty clear that they want the current CIM/WBEM infrastructure fixed and polished. They seem a lot less interested in these fancy new SOAP-based protocols and even less in using any other model than CIM.

It will be interesting to see what this means for new DMTF activities, such as CMDBf or WS-RC, that are supposed to be model-neutral. A few possibilities:

• the priorities of the members change over time to make room for these considerations
• turn-over (or increase) in membership brings in members with a different perspective
• the model-neutral activities slowly get more and more CIM-influenced
• rejection by the DMTF auto-immune system

My guess is that the DMTF leadership is hoping for #1 and/or #2 while the current “base” (to borrow from the US election-season language) wouldn’t mind #3 or #4. I am expecting some mix of #2 and #3.

Pushing the analogy with current US political events further than is reasonable, one can see a correspondence with the Republican primary:

• CIM/WBEM is Huckabe, favored by the base
• CMDBf/WS-RC/WS-Management etc is Romney, the choice of the party leadership
• At the end, some RDF and HTTP-based integration-friendly approach comes from behind and takes the prize (McCain)

Then you still have to win the general election (i.e. industry adoption of whatever the DMTF cooks up).

[UPDATED 2008/2/7: the day after I write this entry, Romney quits the race. Bad omen for CMDBf and WS-RC? ;-) ]

I like XML. Call me crazy but I find it relatively easy to work with. Whether it is hand-editing an XML document in a text editor, manipulating it programmatically (as long as you pick a reasonable API, e.g. XOM in Java), transforming it (e.g. XSLT) or querying an XML back-end through XPath/XQuery. Sure it carries useless features that betray its roots in the publishing world (processing instructions anyone?), sure the whole attribute/element overlap doesn’t have much value for systems modeling, but overall it hits a good compromise between human readability and machine processing and it has a pretty solid extensibility story with namespaces.

In addition, the XML toolbox of specifications is very large and offers standard-based answers to many XML-related tasks. That’s good, but when composing a solution it also means that one needs to keep two things in mind:

• not all these XML specifications are technically sound (even if they carry a W3C stamp of approval), and
• just because XML’s inherent flexibility lets one stretch a round hole, it doesn’t mean it’s a good idea to jam a square peg into it.

The domain of IT management provides examples for both of these risks. These examples constitute some of the technical deficiencies of management-related XML specifications that I mentioned in the previous post. More specifically, let’s look at three instances of XML mis-use that relate to management-related specifications. We will see:

• a terrible XML specification that infects any solution it touches (WS-Addressing, used in WS-Management),
• a mediocre XML specification that has plenty of warts but can be useful for a class of problems, except in this case it isn’t (XSD, used in SML), and
• a very good XML specification except it is used in the wrong place (XPath, used in CMDBf).

Let’s go through them one by one.

WS-Addressing in WS-Management

The main defect of WS-Management (and of WSDM before it) is probably its use of WS-Addressing. SOAP needs WS-Addressing like a migraine patient needs a bullet in the head (actually, four bullets in the head since we got to deal with four successive versions). SOAP didn’t need a new addressing model, it already had URIs. It just needed a message correlation mechanism. But what we got is many useless headers (like wsa:Action) and the awful EPR construct which solves a problem that didn’t exist and creates many very real new ones. One can imagine nifty hacks that would be enabled by a templating mechanism for SOAP (I indulged myself and sketched one to facilicate mash-up style integrations with SOAP) but if that’s what we’re after then there is no reason to limit it to headers.

XSD in SML

The words “Microsoft” and “bully” often appear in the same sentence, but invariably “Microsoft” is the subject not the object of the bullying. Well, to some extent we have a reverse example here, as unlikely as it may seem. Microsoft created an XML-based meta-model called SDM that included capabilities that looked like parts of XSD. When they opened it up to the industry and floated the idea of standardizing it, they heard back pretty loudly that it would have to re-use XSD rather than “re-invent” it. So they did and that ended up as SML. Except it was the wrong choice and in retrospect I think it would have been better to improve on the original SDM to create a management-specific meta-model than swallow XSD (SML does profile out a few of the more obscure features of XSD, like xs:redefine, but that’s marginal). Syntactic validation of documents is very different from validation of IT models. Of course this may all be irrelevant anyway if SML doesn’t get adopted, which at this point still looks like the most likely outcome (due to things like the failure of CML to produce any model element so far, the ever-changing technical strategy for DSI and of course the XSD-induced complexity of SML).

XPath in CMDBf

I have already covered this in my review of CMDBf 1.0. The main problem is that while XML is a fine interchange format for the CMDBf specification, one should not assume that it is the native format of the data stores that get connected. Using XPath as a selector language makes life difficult for those who don’t use XML as their backend format. Especially when it is not just XPath 1.0 but also the much more complex XPath 2.0. To make matters worse, there is no interoperable serialization format for XPath 1.0 nodesets, which will prevent any kind of interoperability on this. That omission can be easily fixed (and I am sure it will be fixed in DMTF) but that won’t address the primary concern. In the context of CMDBf, XPath/XQuery is an excellent implementation choice for some situations, but not something that should be pushed at the level of the protocol. For example, because XPath is based on the XML model, it has clear notions of order of elements. But what if I have an OO or an RDF-based backend? What am I to make of a selector that says that the “foo” element has to come after the “bar” element? There is no notion of order in Java attributes and/or RDF properties.

Revisionism?

My name (in the context of my previous job at HP) appears in all three management specifications listed above (in increasing level of involvement as contributor for WS-Management, co-author for SML and co-editor for CMDBf) so I am not a neutral observer on these questions. My goal here is not to de-associate myself from these specifications or pick and choose the sections I want to be associated with (we can have this discussion over drinks if anyone is interested). Some of these concerns I had at the time the specifications were being written and I was overruled by the majority. Other weren’t as clear to me then as they are now (my view of WS-Addressing has moved over time from “mostly harmless” to “toxic”). I am sure all other authors have a list of things they wished had come out differently. And while this article lists deficiencies of these specifications, I am not throwing the baby with the bathwater. I wrote recently about WS-Management’s potential for providing consistency for resource manageability. I have good hopes for CMDBf, now in the DTMF, not necessarily as a federation technology but as a useful basis for increased interoperability between configuration repositories. SML has the most dubious fate at this time because, unlike the other two, it hasn’t (yet?) transcended its original supporter to become something that many companies clearly see fitting in their plans.

[UPDATED 2008/3/27: For an extreme example of purposely abusing XML technologies (namely XPath in that case) in a scenario in which it is not the right tool for the job (graph queries), check out this XPath brain teasers article.]

It is pretty clear by now that, whether or not it becomes ubiquitous, WS-Management will be around for quite some time as a protocol for resource manageability. Its inclusion in a large number of manageable products with long development cycles (servers, devices, operating systems…) ensures this. But I wonder whether it will also be useful for management integration.

The difference between manageability and management integration may not seem obvious, but it is important. To simplify, a manageability protocol is something that allows you to remotely manage a resource without having to deploy an agent on it. It lets you read the CPU load on a server. It lets you retrieve a list of instances running in a process engine. It lets you reboot a machine. It lets you access the logs of an application. It lets you receive alerts about a resource. Management integration, on the other hand, lets you create management solutions. For example, it’s what you do when you create a management dashboard that presents information aggregated from several management data repositories (e.g. a CMDB, a metrics store and a SOA registry). Or when you run system-wide validation rules to govern a complex system. Or when you perform automated root cause analysis.

Here is another way to illustrate the difference: CIM is useful for manageability. The more recent standardization efforts in the management world (SML, CMDBf) have been focusing on management integration. To some extent, you can even use that difference as the shortest answer to the common question “what is the relationship/difference between SML and CIM”: CIM is designed for manageability and SML for management integration.

The difference between manageability and management integration isn’t alway clear-cut. There are scenarios that could be argued to fall in either category. And management integration scenarios often involve manageability interactions. But if you try to implement management integration scenarios by working at the manageability level, you very quickly get bogged-down. And even if you fight your way to completion, the resulting integration is too brittle to be of any long-term use. You need a level of abstraction over manageability. This is very similar to integration problems in other domains, and this is where SOA comes in, as a design approach to provide resilience and flexibility for management integration. SOA doesn’t help much in manageability scenarios. It can be useful for management integration.

People working on using Web services for management never had a shared understanding of this distinction. If you look at Microsoft’s early scenarios for WS-Management (and their partner list), it is clear that they were focused on manageability, mostly of the Windows OS, the computers it runs on and the devices connected to these computers. On the other hand, when my colleagues at HP Software and I produced WSMF and later worked on WSDM and WS-Management, it was management integration that we cared most about. We didn’t really care much to put a SOAP wrapper around manageability operations. But we understood that this was also happening and it made sense to share tools and expertise between the two sets of scenarios, especially since, as mentioned above, they overlap.

What happened is that manageability is the only place where WS-Management took hold. One reason is that Microsoft was the main force pushing this adoption and this is where they were pushing it. Another is that, with CIM/HTTP and SNMP, the use of standard protocols for manageability was understood (and the prospect of better tools and better alignment with mainstream distributed software technologies was mostly welcomed by that community).

But in my mind, the use of SOAP made by WS-Management is mostly suited for management integration scenarios. In the manageability case, it’s mostly overhead. You don’t really need security beyond what SSL offers. You don’t really need routing through intermediaries. You don’t really need reliable messaging or the flavor of “transactionality” that the WS-* specifications provide. You don’t really need asynchronous messaging. You don’t really need fine-grained get/set operations (when dealing with one resource, operations at the level of the entire representation are often sufficient). Which is why I can’t help shaking my head when I see WS-Management used for manageability and not for management integration. Kind of like using an SUV that can carry eight people over mountains to carry one person to the hairdresser. Crazy, I know.

Leaving the SUV analogy aside, it’s not that WS-Management is perfectly designed for management integration either, not by a long shot. Which takes us to a third reason (and there are more) why WS-Management is not being used in management integration scenarios: it has technical deficiencies as do many of the other specifications recently created for management integration. That’s the topic of the next post.

There is a narrow window of opportunity for WS-Management to become a unifying force that helps lower the need for management agents. Right now, WS-Management is still only “yet another manageability protocol”. Its adoption is growing but there isn’t much you can do with it that you can’t do through some other way (what resources today are only manageable through WS-Management?) and it is not so widely supported that you can get away with supporting just WS-Management.

I see two main reasons keeping pragmatic creators of IT resources (hardware and software) from more widely using WS-Management to expose the manageability capabilities of their resources. The first one, that I will cover here, is the fear of wasting development resources (and the lack of customer demand). The second one, that I will cover in a later post, is the complexity introduced by some technical choices in WS-Management.

There is plenty of uncertainty around the status and future of WS-Management. This means that any investment in implementing the specification is at risk of having to be later thrown away. It also means that customers, while they often mention it as part of a check-list, understand that at this point WS-Management doesn’t necessarily give them the investment protection that widely-supported stable standards provide. And as such they are receptive when vendors explain that at this point there really isn’t a stable standard for manageability that goes across domains and the best they can get is support for a patchwork of established specifications like SNMP, JMX, CIM/HTTP, WMI, etc.

One source of this uncertainty about WS-Management comes from the fact that there is an equivalent standard, WSDM, that came out of OASIS. But at this point, it is pretty clear that WSDM is going nowhere. Good metrics are hard to come by, but if you compare the dates of last commit activity in the three open-source WS-Management implementations that I know of (Openwsman, Wiseman and the WS-Management module of SOA4D) to that of the Muse implementation of WSDM, you are comparing ages in hours/days to ages in months. Another way is to look at the sessions in the Web services track at the recent Management Developers Conference: six presentations around WS-Management (including an intriguing Ruby on Rails module) compared to one for WSDM. Unless your company is an IBM-only account, WSDM isn’t a useful alternative to WS-Management (and it’s not due to technical inferiority, I still prefer WSDM MUWS to WS-Management on that point but it’s largely irrelevant).

The more serious concern is that, back when it wasn’t clear that the industry would pick WS-Management over WSDM, an effort was launched to reconcile the two specifications. That effort, often refered to as the WS-Management/WSDM convergence, is private so no-one outside of the four companies involved know what is happening. The only specification that has come out at this point is a draft of WS-ResourceTransfer in summer 2006 (I don’t include WS-ResourceCatalog because even though it came out of the same group it provides features that are neither in WS-Management nor in WSDM so it is not really part of converging them). What is happening now? The convergence effort may have died silently. Or it may be on the brink of releasing a complete new set of specifications. Or it may have focused on a more modest set of enhancements to WS-Management. Even though I was in the inside until a few months ago, I am not feigning ignorance here. There is enough up in the air that I can visualize any of these options realized.

This is not encouraging to people looking to invest their meager development resources to improve manageability interfaces on their products. What if they put work in WS-Management and soon after that Microsoft, IBM, HP and Intel come out with a new set of specifications and try to convince the industry to move from WS-Management to that new set of specifications? Much safer to stay on the sidelines for now. The convergence is a source of FUD preventing adoption of WS-Management. It is, on the other hand, a lifeline for WSDM because it provides a reason for those who went with WSDM to wait and see what happens with the convergence before moving away from WSDM.

Even before leaving HP, I had come to the conclusion that it was too late for the convergence to succeed. This doesn’t imply anything about HP’s current position on the topic, which I am of course not qualified to represent. But I just noticed that the new HP BTO chief architect doesn’t seem too fond of WS-*.

Even if the convergence effort manages to deliver the specifications it promised (including an update of WS-ResourceTransfer which is currently flawed, especially its “partial put” functionality), it will be years before they get published, interop-tested, submitted and standardized. Will there be appetite for a new set of WS-* specifications at that point? Very doubtful. SOAP will be around for a long time, but the effort in the SOAP community is around using the existing set of specifications to address already-identified enterprise integration problems. The final stage in the production of any good book, article or even blog post (not that this blog is a shining example) is to pair-down the content, to remove anything that is not essential. This is the stage that the SOAP world is in, sorting through the deluge of specifications to extract and polish the productive core. New multi-spec frameworks need not apply.

If there is to emerge a new, comprehensive, framework for web-based manageability, it won’t be the WS-Management/WSDM convergence. It probably won’t use SOAP (or at least not in its WS-Addressing-infected form). It may well use RDF. But it is not in sight at this point. So for now the choice is whether to seize the opportunity to create a widely-adopted standard on the basis of WS-Management (with all its flaws) or to let the window of opportunity close, to treat WS-Management as just another manageability tool in the toolbox and go on with life. Until the stars line up in a few years and the industry can maybe take another stab at the effort. To a large extent, this is in the hands of Microsoft, IBM, HP and Intel. Ironically, the best way for those who want nothing to do with SOAP to prevent SOAP from being used too much for manageability (beyond where WS-Management is already used) is to keep pushing the convergence (which is very much SOAP based) in order to keep WS-Management contained.