William Vambenepe's blog

There is a narrow window of opportunity for WS-Management to become a unifying force that helps lower the need for management agents. Right now, WS-Management is still only “yet another manageability protocol”. Its adoption is growing but there isn’t much you can do with it that you can’t do through some other way (what resources today are only manageable through WS-Management?) and it is not so widely supported that you can get away with supporting just WS-Management.

I see two main reasons keeping pragmatic creators of IT resources (hardware and software) from more widely using WS-Management to expose the manageability capabilities of their resources. The first one, that I will cover here, is the fear of wasting development resources (and the lack of customer demand). The second one, that I will cover in a later post, is the complexity introduced by some technical choices in WS-Management.

There is plenty of uncertainty around the status and future of WS-Management. This means that any investment in implementing the specification is at risk of having to be later thrown away. It also means that customers, while they often mention it as part of a check-list, understand that at this point WS-Management doesn’t necessarily give them the investment protection that widely-supported stable standards provide. And as such they are receptive when vendors explain that at this point there really isn’t a stable standard for manageability that goes across domains and the best they can get is support for a patchwork of established specifications like SNMP, JMX, CIM/HTTP, WMI, etc.

One source of this uncertainty about WS-Management comes from the fact that there is an equivalent standard, WSDM, that came out of OASIS. But at this point, it is pretty clear that WSDM is going nowhere. Good metrics are hard to come by, but if you compare the dates of last commit activity in the three open-source WS-Management implementations that I know of (Openwsman, Wiseman and the WS-Management module of SOA4D) to that of the Muse implementation of WSDM, you are comparing ages in hours/days to ages in months. Another way is to look at the sessions in the Web services track at the recent Management Developers Conference: six presentations around WS-Management (including an intriguing Ruby on Rails module) compared to one for WSDM. Unless your company is an IBM-only account, WSDM isn’t a useful alternative to WS-Management (and it’s not due to technical inferiority, I still prefer WSDM MUWS to WS-Management on that point but it’s largely irrelevant).

The more serious concern is that, back when it wasn’t clear that the industry would pick WS-Management over WSDM, an effort was launched to reconcile the two specifications. That effort, often refered to as the WS-Management/WSDM convergence, is private so no-one outside of the four companies involved know what is happening. The only specification that has come out at this point is a draft of WS-ResourceTransfer in summer 2006 (I don’t include WS-ResourceCatalog because even though it came out of the same group it provides features that are neither in WS-Management nor in WSDM so it is not really part of converging them). What is happening now? The convergence effort may have died silently. Or it may be on the brink of releasing a complete new set of specifications. Or it may have focused on a more modest set of enhancements to WS-Management. Even though I was in the inside until a few months ago, I am not feigning ignorance here. There is enough up in the air that I can visualize any of these options realized.

This is not encouraging to people looking to invest their meager development resources to improve manageability interfaces on their products. What if they put work in WS-Management and soon after that Microsoft, IBM, HP and Intel come out with a new set of specifications and try to convince the industry to move from WS-Management to that new set of specifications? Much safer to stay on the sidelines for now. The convergence is a source of FUD preventing adoption of WS-Management. It is, on the other hand, a lifeline for WSDM because it provides a reason for those who went with WSDM to wait and see what happens with the convergence before moving away from WSDM.

Even before leaving HP, I had come to the conclusion that it was too late for the convergence to succeed. This doesn’t imply anything about HP’s current position on the topic, which I am of course not qualified to represent. But I just noticed that the new HP BTO chief architect doesn’t seem too fond of WS-*.

Even if the convergence effort manages to deliver the specifications it promised (including an update of WS-ResourceTransfer which is currently flawed, especially its “partial put” functionality), it will be years before they get published, interop-tested, submitted and standardized. Will there be appetite for a new set of WS-* specifications at that point? Very doubtful. SOAP will be around for a long time, but the effort in the SOAP community is around using the existing set of specifications to address already-identified enterprise integration problems. The final stage in the production of any good book, article or even blog post (not that this blog is a shining example) is to pair-down the content, to remove anything that is not essential. This is the stage that the SOAP world is in, sorting through the deluge of specifications to extract and polish the productive core. New multi-spec frameworks need not apply.

If there is to emerge a new, comprehensive, framework for web-based manageability, it won’t be the WS-Management/WSDM convergence. It probably won’t use SOAP (or at least not in its WS-Addressing-infected form). It may well use RDF. But it is not in sight at this point. So for now the choice is whether to seize the opportunity to create a widely-adopted standard on the basis of WS-Management (with all its flaws) or to let the window of opportunity close, to treat WS-Management as just another manageability tool in the toolbox and go on with life. Until the stars line up in a few years and the industry can maybe take another stab at the effort. To a large extent, this is in the hands of Microsoft, IBM, HP and Intel. Ironically, the best way for those who want nothing to do with SOAP to prevent SOAP from being used too much for manageability (beyond where WS-Management is already used) is to keep pushing the convergence (which is very much SOAP based) in order to keep WS-Management contained.

It’s now official, the CMDBf specification has been submitted to the DMTF and will be standardized there. Here is the press release and here is the specification (unchanged) republished on the DMTF site. The CMDBf working group was created a while ago at the DMTF but I didn’t report it since it wasn’t clear to me whether that was public information or not. The press release makes this clear now.

As a side note, this is one of my ongoing frustrations with the DMTF. Almost everything happens in private with no publicly-accessible URL until a press release comes out and of course lots of interesting things happen that don’t get a press release. I have heard many times that the DMTF is working on opening up the process, but I still haven’t seen much change. If this had been OASIS or W3C, the call for formation of the new working group would have been publicly accessible even before the group was created. OK, end of ranting.

As always, there isn’t much useful information to be gleaned from the text of the press release. Only that, as expected, the authors addressed the question of how this relates to CIM, since for many DMTF=CIM. So the press release proactively declares that the CMDBf work will not be limited to CIM-modeled configuration data. What this means in practice will be seen later (e.g. will there be CIM-specific extensions?).

Having seen how executive quotes for press releases get generated I hate to read too much into them, but another thing I can’t help noticing in the press release is that none of the quotes from the companies submitting the specification tout federation, but simply “integration” or “sharing”. For example: “integration and interoperability” (BMC), “share data” (CA), “sharing of information” (HP), “view, track and change information” (IBM), “exchange data” (Microsoft). This more realistic assessment of what the specification does stands in contrast to the way the DMTF presents it in the press release : “this specification provides a standard way to federate management data stored in multiple different data models”. At this point, it doesn’t really provide federation and especially not across different models.

All in all, it’s as good thing for this work to be moved to a standards organization. I may join the CMDBf group at the DMTF to track it, but I don’t plan to engage very much as this area isn’t my focus anymore now that I am at Oracle. But of course everything is linked at some level in the management field.

[UPDATE  on 2007/11/30: two days after posting this message I got the monthly DMTF newsletter which touches on points I raise here. So here are the relevant links. First, Mike Baskey, DMTF Chairman, shares his view on what CMDBf means for DMTF. Second, as if to respond to my rant on the opacity of the DMTF, Josh Cohen, DMTF Vice-chairman, gives an update on process improvements. Some progress indeed, but still a far cry from opening up mailing list archives so that observers can see in real time what issues are addressed and can go back in time to understand how a specific technical decision was made and what were the considerations.]

I finally took a look at OVF, the virtual machine distribution specification that was recently submitted to DMTF. The document is authored by VMware and XenSource, but they are joined in the submission to DMTF by some other biggies, namely Microsoft, HP, IBM and Dell.

Overall, the specification does a good job of going after the low-hanging fruits of VM distribution/portability. And the white paper is very good. I wish I could say that all the specifications I have been associated with came accompanied by such a clear description of what they are about.

I am not a virtualization, operating system or hardware expert. I am mostly looking at this specification from the systems management perspective. More specifically I see virtualization and standardization as two of the many threads that create a great opportunity for increased automation of IT management and more focus on the application rather than the infrastructure (which is part of why I am now at Oracle). Since OVF falls in both the “virtualization” and “standardization” buckets, it got my attention. And the stated goal of the specification (”facilitate the automated, secure management not only of virtual machines but the appliance as a functional unit”, see section 3.1) seems to fit very well with this perspective.

On the other hand, the authors explicitly state that in the first version of the specification they are addressing the package/distribution stage and the deployment stage, not the earlier stage (development) or the later ones (management and retirement). This sidesteps many of the harder issues, which is part of why I write that the specification goes after the low-hanging fruits (nothing wrong with starting that way BTW).

The other reason for the “low hanging fruit” statement is that OVF is just a wrapper around proprietary virtual disk formats. It is not a common virtual disk format. I’ve read in several news reports that this specification provides portability across VM platforms. It’s sad but almost expected that the IT press would get this important nuance wrong, it’s more disappointing when analysts (who should know better) do, as for example the Burton Group which writes in its analysis “so when OVF is supported on Xen and VMware virtualization platforms for example, a VM packaged on a VMware hypervisor can run on a Xen hypervisor, and vice-versa”. That’s only if someone at some point in the chain translates from the Xen virtual disk format to the VMware one. OVF will provide deployment metadata and will allow you to package both virtual disks in a TAR if you so desire, but it will not do the translation for you. And the OVF authors are pretty up front about this (for example, the white paper states that “the act of packaging a virtual machine into an OVF package does not guarantee universal portability or install-ability across all hypervisors”). On a side note, this reminds me a bit of how the Sun/Microsoft Web SSO MEX and Web SSO Interop Profile specifications were supposed to bridge Passport with WS-Federation which was a huge overstatement. Except that in that case, the vendors were encouraging the misconception (which the IT press happily picked up) while in the OVF case it seems like the vendors are upfront about the limitations.

There is nothing rocket-science about OVF and even as a non-virtualization expert it makes sense to me. I was very intrigued by the promise that the specification “directly supports the configuration of multi-tier applications and the composition of virtual machines to deliver composed services” but this turns out to be a bit of an overstatement. Basically, you can distribute the VMs across networks by specifying a network name for each VM. I can easily understand the simple case, where all the VMs are on the same network and talking to one another. But there is no way (that I can see) to specify the network topology that joins different networks together, e.g. saying that there is a firewall between networks “blue” and “red” that only allows traffic on port 80). So why would I create an OVF file that composes several virtual machines if they are going to be deployed on networks that have no relationships to one another? I guess the one use case I can think of would be if one of the virtual machines was assigned to two networks and acted as a gateway/firewall between them. But that’s not a very common and scalable way to run your networks. There is a reason why Cisco sells $30 billions of networking gear every year. So what’s the point of this lightweight distributed deployment? Is it just for that use case where the network gear is also virtualized, in the expectation of future progress in that domain? Is this just a common anchor point to be later extended with more advanced network topology descriptions? This looks to me like an attempt to pick a low-hanging fruit that wasn’t there.

Departing from usual practice, this submission doesn’t seem to come with any license grant, which must have greatly facilitated its release and the recruitment of supporters for the submission. But it should be a red flag for adopters. It’s worth keeping track of its IP status as the work progresses. Unless things have changed recently, DMTF’s IP policy is pretty weak so the fact that works happens there doesn’t guarantee much protection per se to the adopters. Interestingly, there are two sections (6.2 about the virtual disk format and 11.3 about the communication between the guest software and the deployment platform) where the choice of words suggests the intervention of patent lawyers: phrases like “unencumbered specification” (presumably unencumbered with licensing requirements) and “someone skilled in the art”. Which is not surprising since this is the part where the VMWare-specific, Xen-specific or Microsoft-specific specifications would plug in.

Speaking of lawyers, the section that allows the EULA to be shipped with the virtual appliance is very simplistic. It’s just a human-readable piece of text in the OVF file. The specification somewhat naively mentions that “if unattended installs are allowed, all embedded license sections are implicitly accepted”. Great, thanks, enterprises love to implicitly accept licensing terms. I would hope that the next version will provide, at least, a way to have a URI to identify the EULA so that I can maintain a list of pre-approved EULAs for which unattended deployment is possible. Automation of IT management is supposed to makes things faster and cheaper. Having a busy and expensive lawyer read a EULA as part of my deployment process goes against both objectives.

It’s nice of the authors to do the work of formatting the specification using the DMTF-approved DSPxxxx format before submitting to the organization. But using a targetnamespace in the dmtf.org domain when the specification is just a submission seems pretty tacky to me, unless they got a green light from the DMTF ahead of time. Also, it looks a little crass on the part of VMware to wrap the specification inside their corporate white paper template (cover page and back page) if this is a joint publication. See the links at http://www.vmware.com/appliances/learn/ovf.html. Even though for all I know VMware might have done most of the actual work. That’s why the links that I used to the white paper and the specification are those at XenSource, which offers the plain version. But then again, this specification is pretty much a wrapper around a virtual disk file, so graphically wrapping it may have seemed appropriate…

OK, now for some XML nitpicking.

I am not a fan of leaving elementformdefault set to “unqualified” but it’s their right to do so. But then they qualify all the attributes in the specification examples. That looks a little awkward to me (I tend to do the opposite and qualify the elements but not the attributes) and, more importantly, it violates the schema in appendix since the schema leaves attributeFormDefault to its default value (unqualified). I would rather run a validation before makings this accusation, but where are the stand-alone XSD files? The white paper states that “it is the intention of the authors to ensure that the first version of the specification is implemented in their products, and so the vendors of virtual appliances and other ISV enablement, can develop to this version of the specification” but do you really expect us to copy/paste from PDF and then manually remove the line numbers and header/footer content that comes along? Sorry, I have better things to do (like whine about it on this blog) so I haven’t run the validation to verify that the examples are indeed in violation. But that’s at least how they look to me.

I also have a problem with the Section and Content elements that are just shells defined by the value of their xsi:type attribute. The authors claim it’s for extensibility (”the use of xsi:type is a core part of making the OVF extensible, since additional type definitions for sections can be added”) but there are better ways to do extensibility in XML (remember, that’s what the X stands for). It would be better to define an element per type (disk, network…). They could possibly be based on the same generic type in XSD. And this way you get more syntactic flexibility and you get the option to have sub-types of sub-types rather than a flat list. Interestingly, there is a comment inside the XSD that defines the Section type that reads “the base class for a section. Subclassing this is the most common form of extensibility”. That’s the right approach, but somehow it got dropped at some point.

Finally, the specification seems to have been formated based on WS-Management (which is the first specification that mixed the traditional WS-spec conventions with the DMTF DSPxxxx format), which may explain why WS-Management is listed as a reference at the end even though it is not used anywhere in the specification. That’s fine but it shows in a few places where more editing is needed. For example requirement R1.5-1 states that “conformant services of this specification MUST use this XML namespace Universal Resource Identifier (URI): http://schemas.dmtf.org/ovf”. I know what a conformant service is for WS-Management but I don’t know what it is for this specification. Also, the namespace that this requirement uses is actually not defined or used by this specification, so this requirement is pretty meaningless. The table of namespaces that follows just after is missing some namespaces. For example, the prefix “xsi” is used on line 457 (xsi:any and xsi:AnyAttribute) and I want to say it’s the wrong one as xsi is usually assigned to “http://www.w3.org/2001/XMLSchema-instance” and not “http://www.w3.org/2001/XMLSchema” but since the prefix is not in the table I guess it’s anyone’s guess (and BTW, it’s “anyAttribute”, not “AnyAttribute”).

By this point I may sound like I don’t like the specification. Not at all. I still stand with what I wrote in the second paragraph. It’s a good specification and the subset of problems that it addresses is a useful subset. There are a few things to fix in the current content and several more specifications to write to complement it, but it’s a very good first step and I am glad to see VMware and XenSource collaborating on this. Microsoft is nominally in support at this point, but it remains to be seen to what extent. I haven’t seen them in the past very interested in standards effort that they are not driving and so far this doesn’t appear to be something they are driving.