Category Archives: IT Systems Mgmt

January 27, 2008

IT management in a world of utility IT

A cynic might call it “could computing” rather than “cloud computing”. What if you could get rid of your data center. What if you could pay only for what you use. What if you could ramp up your capacity on the fly. We’ve been hearing these promising pitches for a while now and recently the intensity has increased, fueled by some real advances.

As an IT management architect who is unfortunately unlikely to be in position to retire anytime soon (donations accepted for the send-William-to-retirement-on-a-beach fund) it forces me to wonder what IT management would look like in a world in which utility computing is a common reality.

First, these utility computing providers themselves will need plenty of IT management, if not necessarily the exact same kind that is being sold to enterprises today. You still need provisioning (automated of course). You definitely need access measuring and billing. Disaster recovery. You still have to deal with change planning, asset management and maybe portfolio management. You need processes and tools to support them. Of course you still have to monitor, manage SLAs, and pinpoints problems and opportunities for improvement. Etc. Are all of these a source of competitive advantage? Google is well-known for writing its infrastructure software (and of course also its applications) in house but there is no reason it should be that way, especially as the industry matures. Even when your business is to run a data center, not all aspects of IT management provide competitive differentiation. It is also very unclear at this point what the mix will be of utility providers that offer raw infrastructure (like EC2/S3) versus applications (like CRM as a service), a difference that may change the scope of what they would consider their crown jewels.

An important variable in determining the market for IT management software directed at utility providers is the number of these providers. Will there be a handful or hundreds? Many people seem to assume a small number, but my intuition goes the other way. The two main reasons for being only a handful would be regulation and infrastructure limitations. But, unlike with today’s utilities, I don’t see either taking place for utility computing (unless you assume that the network infrastructure is going to get vertically integrated in the utility data center offering). The more independent utility computing providers there are, the more it makes sense for them to pool resources (either explicitly through projects like the Collaborative Software Initiative or implicitly by buying from the same set of vendors) which creates a market for IT management products for utility providers. And conversely, the more of a market offering there is for the software and hardware building blocks of a utility computing provider, the lower the economies of scale (e.g. in software development costs) that would tend to concentrate the industry.

Oracle for one is already selling to utility providers (SaaS-type more than EC2-type at this point) with solutions that address scalability, SLA and multi-tenancy. Those solutions go beyond the scope of this article (they include not just IT management software but also databases and applications) but Oracle Enterprise Manager for IT management is also part of the solution. According to this Aberdeen report the company is doing very well in that market.

The other side of the equation is the IT management software that is needed by the consumers of utility computing. Network management becomes even more important. Identity/security management. Desktop management of some sort (depending on whether and what kind of desktop virtualization you use). And, as Microsoft reminds us with S+S, you will most likely still be running some software on-premises that needs to be managed (Carr agrees). The new, interesting thing is going to be the IT infrastructure to manage your usage of utility computing services as well as their interactions with your in-house software. Which sounds eerily familiar. In the early days of WSMF, one of the scenarios we were attempting to address (arguably ahead of the times) was service management across business partners (that is, the protocols and models were supposed to allow companies to expose some amount of manageability along with the operational services, so that service consumers would be able to optimize their IT management decision by taking into account management aspects of the consumed services). You can see this in the fact that the WSMF-WSM specification (that I co-authored and edited many years ago at HP) contains a model of a “conversation” that represents “set of related messages exchanged with other Web services” (a decentralized view of a BPEL instance, one that represents just one service’s view of its participation in the instance). Well, replace “business partner” with “SaaS provider” and you’re in a very similar situation. If my business application calls a mix of internal services, SaaS-type services and possibly some business partner services, managing SLAs and doing impact/root cause analysis works a lot better if you get some management information from these other services. Whether it is offered by the service owner directly, by a proxy/adapter that you put on your end or by a neutral third party in charge of measuring/enforcing SLAs. There are aspects of this that are “regular” SOA management challenges (i.e. that apply whenever you compose services, whether you host them yourself or not) and there are aspects (security, billing, SLA, compliance, selection of partners, negotiation) that are handled differently in the situation where the service is consumed from a third party. But by and large, it remains a problem of management integration in a word of composed, orchestrated and/or distributed applications. Which is where it connects with my day job at Oracle.

Depending on the usage type and the level of industry standardization, switching from one utility computing provider to the other may be relatively painless and easy (modify some registry entries or some policy or even let it happen automatically based on automated policies triggered by a price change for example) or a major task (transferring huge amounts of data, translating virtual machines from one VM format to another, performing in-depth security analysis…). Market realities will impact the IT tools that get developed and the available IT tools will in return shape the market.

Another intriguing opportunity, if you assume a mix of on-premises computing and utility-based computing, is that of selling back your spare capacity on the grid. That too would require plenty of supporting IT management software for provisioning, securing, monitoring and policing (coming soon to an SEC filing: “our business was hurt by weak sales of our flagship Pepsi cola drink, partially offset by revenue from renting computing power from our data center to the Coca cola company to handle their exploding ERP application volume”). I believe my neighbors with solar panels on their roofs are able to run their electric counter backward and sell power to PG&E when they generate more than they use. But I’ll stop here with the electric grid analogy because it is already overused. I haven’t read Carr’s book so the comment may be unfair, but based on extracts he posted and reviews he seems to have a hard time letting go of that analogy. It does a good job of making the initial point but gets tiresome after a while. Having personally experienced the Silicon Valley summer rolling black-outs, I very much hope the economics of utility computing won’t be as warped. For example, I hope that the telcos will only act as technical, not commercial intermediaries. One of the many problems in California is that the consumer don’t buy from the producers but from a distributor (PG&E in the Bay Area) who sells at a fixed price and then has to buy at pretty much any price from the producers and brokers who made a killing manipulating the supply during these summers. Utility computing is another area in which economics and technology are intrinsically and dynamically linked in a way that makes predictions very difficult.

For those not yet bored of this topic (or in search of a more insightful analysis), Redmonk’s Coté has taken a crack at that same question, but unlike me he stays clear of any amateurish attempt at an economic analysis. You may also want to read Ian Foster’s analysis (interweaving pieces of technology, standards, economy, marketing, computer history and even some movie trivia) on how these “clouds” line up with the “grids” that he and others have been working on for a while now. Some will see his post as a welcome reminder that the only thing really new in “cloud” computing is the name and others will say that the other new thing is that it is actually happening in a way that matters to more than a few academics and that Ian is just trying to hitch his jalopy to the express train that’s passing him. For once I am in the “less cynical” camp on this and I think a lot of the “traditional” Grid work is still very relevant. Did I hear “EC2 components for SmartFrog”?

[UPDATED 2008/6/30: For a comparison of “cloud” and “grid”, see here.]

[UPDATED 2008/9/22: More on the Cloud vs. Grid debate: a paper critical of Grid (in the OGF sense of the term) efforts and Ian Foster’s reply (reat the comments too).]

11 Comments

Filed under Business, Everything, IT Systems Mgmt, Utility computing, Virtualization

December 25, 2007

Top 10 lists and virtualization management

Over the last few months, I have seen two “top 10” lists with almost the same title and nearly zero overlap in content. One is Network World’s “10 virtualization companies to watch” published in August 2007. The other is CIO’s “10 Virtualization Vendors to Watch in 2008” published three months later. To be precise, there is only one company present in both lists, Marathon Technologies. Congratulations to them (note to self: hire their PR firm when I start my own company). Things are happening quickly in that field, but I doubt the landscape changed drastically in these three months (even though the announcement of Oracle’s Virtual Machine product came during that period). So what is this discrepancy telling us?

If anything, this is a sign of the immaturity of the emerging ecosystem around virtualization technologies. That being said, it could well be that all this really reflects is the superficiality of these “top 10” lists and the fact that they measure PR efforts more than any market/technology fact (note to self: try to become less cynical in 2008) (note to self: actually, don’t).

So let’s not read too much into the discrepancy. Less striking but more interesting is the fact that these lists are focused on management tools rather than hypervisors. It is as if the competitive landscape for hypervisors was already defined. And, as shouldn’t be a surprise, it is defined in a way that closely mirrors the operating system landscape, with Xen as Linux (the various Xen-based offerings correspond to the Linux distributions), VMWare as Solaris (good luck) and Microsoft as, well Microsoft.

In the case of Windows and Hyper-V, it is actually bundled as one product. We’ll see this happen more and more on the Linux/Xen side as well, as illustrated by Oracle’s offering. I wouldn’t be surprised to see this bundling so common that people start to refer to it as “LinuX” with a capital X.

Side note: I tried to see if the word “LinuX” is already being used but neither Google nor Yahoo nor MSN seems to support case-sensitive searching. From the pre-Google days I remember that Altavista supported it (a lower-case search term meant “any capitalization”, any upper-case letter in the search term meant “this exact capitalization”) but they seem to have dropped it too. Is this too computationally demanding at this scale? Is there no way to do a case-sensitive search on the Web?

With regards to management tools for virtualized environments, I feel pretty safe in predicting that the focus will move from niche products (like those on these lists) that deal specifically with managing virtualization technology to the effort of managing virtual entities in the context of the overall IT management effort. Just like happened with security management and SOA management. And of course that will involve the acquisition of some of the niche players, for which they are already positioning themselves. The only way I could be proven wrong on such a prediction is by forecasting a date, so I’ll leave it safely open ended…

As another side note, since I mention Network World maybe I should disclose that I wrote a couple of articles for them (on topics like model-based management) in the past. But when filtering for bias on this blog it’s probably a lot more relevant to keep in mind that I am currently employed by Oracle than to know what journal/magazine I’ve been published in.

Comments Off on Top 10 lists and virtualization management

Filed under Everything, IT Systems Mgmt, Linux, Microsoft, Oracle, OVM, Tech, Virtualization, VMware, XenSource

December 19, 2007

How not to re-use XML technologies

I like XML. Call me crazy but I find it relatively easy to work with. Whether it is hand-editing an XML document in a text editor, manipulating it programmatically (as long as you pick a reasonable API, e.g. XOM in Java), transforming it (e.g. XSLT) or querying an XML back-end through XPath/XQuery. Sure it carries useless features that betray its roots in the publishing world (processing instructions anyone?), sure the whole attribute/element overlap doesn’t have much value for systems modeling, but overall it hits a good compromise between human readability and machine processing and it has a pretty solid extensibility story with namespaces.

In addition, the XML toolbox of specifications is very large and offers standard-based answers to many XML-related tasks. That’s good, but when composing a solution it also means that one needs to keep two things in mind:

not all these XML specifications are technically sound (even if they carry a W3C stamp of approval), and
just because XML’s inherent flexibility lets one stretch a round hole, it doesn’t mean it’s a good idea to jam a square peg into it.

The domain of IT management provides examples for both of these risks. These examples constitute some of the technical deficiencies of management-related XML specifications that I mentioned in the previous post. More specifically, let’s look at three instances of XML mis-use that relate to management-related specifications. We will see:

a terrible XML specification that infects any solution it touches (WS-Addressing, used in WS-Management),
a mediocre XML specification that has plenty of warts but can be useful for a class of problems, except in this case it isn’t (XSD, used in SML), and
a very good XML specification except it is used in the wrong place (XPath, used in CMDBf).

Let’s go through them one by one.

WS-Addressing in WS-Management

The main defect of WS-Management (and of WSDM before it) is probably its use of WS-Addressing. SOAP needs WS-Addressing like a migraine patient needs a bullet in the head (actually, four bullets in the head since we got to deal with four successive versions). SOAP didn’t need a new addressing model, it already had URIs. It just needed a message correlation mechanism. But what we got is many useless headers (like wsa:Action) and the awful EPR construct which solves a problem that didn’t exist and creates many very real new ones. One can imagine nifty hacks that would be enabled by a templating mechanism for SOAP (I indulged myself and sketched one to facilicate mash-up style integrations with SOAP) but if that’s what we’re after then there is no reason to limit it to headers.

XSD in SML

The words “Microsoft” and “bully” often appear in the same sentence, but invariably “Microsoft” is the subject not the object of the bullying. Well, to some extent we have a reverse example here, as unlikely as it may seem. Microsoft created an XML-based meta-model called SDM that included capabilities that looked like parts of XSD. When they opened it up to the industry and floated the idea of standardizing it, they heard back pretty loudly that it would have to re-use XSD rather than “re-invent” it. So they did and that ended up as SML. Except it was the wrong choice and in retrospect I think it would have been better to improve on the original SDM to create a management-specific meta-model than swallow XSD (SML does profile out a few of the more obscure features of XSD, like xs:redefine, but that’s marginal). Syntactic validation of documents is very different from validation of IT models. Of course this may all be irrelevant anyway if SML doesn’t get adopted, which at this point still looks like the most likely outcome (due to things like the failure of CML to produce any model element so far, the ever-changing technical strategy for DSI and of course the XSD-induced complexity of SML).

XPath in CMDBf

I have already covered this in my review of CMDBf 1.0. The main problem is that while XML is a fine interchange format for the CMDBf specification, one should not assume that it is the native format of the data stores that get connected. Using XPath as a selector language makes life difficult for those who don’t use XML as their backend format. Especially when it is not just XPath 1.0 but also the much more complex XPath 2.0. To make matters worse, there is no interoperable serialization format for XPath 1.0 nodesets, which will prevent any kind of interoperability on this. That omission can be easily fixed (and I am sure it will be fixed in DMTF) but that won’t address the primary concern. In the context of CMDBf, XPath/XQuery is an excellent implementation choice for some situations, but not something that should be pushed at the level of the protocol. For example, because XPath is based on the XML model, it has clear notions of order of elements. But what if I have an OO or an RDF-based backend? What am I to make of a selector that says that the “foo” element has to come after the “bar” element? There is no notion of order in Java attributes and/or RDF properties.

Revisionism?

My name (in the context of my previous job at HP) appears in all three management specifications listed above (in increasing level of involvement as contributor for WS-Management, co-author for SML and co-editor for CMDBf) so I am not a neutral observer on these questions. My goal here is not to de-associate myself from these specifications or pick and choose the sections I want to be associated with (we can have this discussion over drinks if anyone is interested). Some of these concerns I had at the time the specifications were being written and I was overruled by the majority. Other weren’t as clear to me then as they are now (my view of WS-Addressing has moved over time from “mostly harmless” to “toxic”). I am sure all other authors have a list of things they wished had come out differently. And while this article lists deficiencies of these specifications, I am not throwing the baby with the bathwater. I wrote recently about WS-Management’s potential for providing consistency for resource manageability. I have good hopes for CMDBf, now in the DTMF, not necessarily as a federation technology but as a useful basis for increased interoperability between configuration repositories. SML has the most dubious fate at this time because, unlike the other two, it hasn’t (yet?) transcended its original supporter to become something that many companies clearly see fitting in their plans.

[UPDATED 2008/3/27: For an extreme example of purposely abusing XML technologies (namely XPath in that case) in a scenario in which it is not the right tool for the job (graph queries), check out this XPath brain teasers article.]

4 Comments

Filed under CMDB Federation, CMDBf, Everything, IT Systems Mgmt, Microsoft, SML, SOAP, SOAP header, Specs, Standards, Tech, WS-Management, XOM

December 14, 2007

Manageability, management integration and WS-Management

It is pretty clear by now that, whether or not it becomes ubiquitous, WS-Management will be around for quite some time as a protocol for resource manageability. Its inclusion in a large number of manageable products with long development cycles (servers, devices, operating systems…) ensures this. But I wonder whether it will also be useful for management integration.

The difference between manageability and management integration may not seem obvious, but it is important. To simplify, a manageability protocol is something that allows you to remotely manage a resource without having to deploy an agent on it. It lets you read the CPU load on a server. It lets you retrieve a list of instances running in a process engine. It lets you reboot a machine. It lets you access the logs of an application. It lets you receive alerts about a resource. Management integration, on the other hand, lets you create management solutions. For example, it’s what you do when you create a management dashboard that presents information aggregated from several management data repositories (e.g. a CMDB, a metrics store and a SOA registry). Or when you run system-wide validation rules to govern a complex system. Or when you perform automated root cause analysis.

Here is another way to illustrate the difference: CIM is useful for manageability. The more recent standardization efforts in the management world (SML, CMDBf) have been focusing on management integration. To some extent, you can even use that difference as the shortest answer to the common question “what is the relationship/difference between SML and CIM”: CIM is designed for manageability and SML for management integration.

The difference between manageability and management integration isn’t alway clear-cut. There are scenarios that could be argued to fall in either category. And management integration scenarios often involve manageability interactions. But if you try to implement management integration scenarios by working at the manageability level, you very quickly get bogged-down. And even if you fight your way to completion, the resulting integration is too brittle to be of any long-term use. You need a level of abstraction over manageability. This is very similar to integration problems in other domains, and this is where SOA comes in, as a design approach to provide resilience and flexibility for management integration. SOA doesn’t help much in manageability scenarios. It can be useful for management integration.

People working on using Web services for management never had a shared understanding of this distinction. If you look at Microsoft’s early scenarios for WS-Management (and their partner list), it is clear that they were focused on manageability, mostly of the Windows OS, the computers it runs on and the devices connected to these computers. On the other hand, when my colleagues at HP Software and I produced WSMF and later worked on WSDM and WS-Management, it was management integration that we cared most about. We didn’t really care much to put a SOAP wrapper around manageability operations. But we understood that this was also happening and it made sense to share tools and expertise between the two sets of scenarios, especially since, as mentioned above, they overlap.

What happened is that manageability is the only place where WS-Management took hold. One reason is that Microsoft was the main force pushing this adoption and this is where they were pushing it. Another is that, with CIM/HTTP and SNMP, the use of standard protocols for manageability was understood (and the prospect of better tools and better alignment with mainstream distributed software technologies was mostly welcomed by that community).

But in my mind, the use of SOAP made by WS-Management is mostly suited for management integration scenarios. In the manageability case, it’s mostly overhead. You don’t really need security beyond what SSL offers. You don’t really need routing through intermediaries. You don’t really need reliable messaging or the flavor of “transactionality” that the WS-* specifications provide. You don’t really need asynchronous messaging. You don’t really need fine-grained get/set operations (when dealing with one resource, operations at the level of the entire representation are often sufficient). Which is why I can’t help shaking my head when I see WS-Management used for manageability and not for management integration. Kind of like using an SUV that can carry eight people over mountains to carry one person to the hairdresser. Crazy, I know.

Leaving the SUV analogy aside, it’s not that WS-Management is perfectly designed for management integration either, not by a long shot. Which takes us to a third reason (and there are more) why WS-Management is not being used in management integration scenarios: it has technical deficiencies as do many of the other specifications recently created for management integration. That’s the topic of the next post.

[UPDATED 2009/6/26: EMC’s Chuck Hollis explains “management versus manageability” (he calls management “service orchestration” and manageability “element management”) in a much simpler way than I was able to. And he hints at upcoming management orchestration software from EMC (time will tell whether they missed out on BladeLogic and Opsware or made the right choice to let others acquire them). It will be interesting to see which of the 7 roads to IT automation middleware they take.]

4 Comments

Filed under Everything, IT Systems Mgmt, Manageability, Mgmt integration, SOAP, WS-Management

December 13, 2007

The window of opportunity for WS-Management

There is a narrow window of opportunity for WS-Management to become a unifying force that helps lower the need for management agents. Right now, WS-Management is still only “yet another manageability protocol”. Its adoption is growing but there isn’t much you can do with it that you can’t do through some other way (what resources today are only manageable through WS-Management?) and it is not so widely supported that you can get away with supporting just WS-Management.

I see two main reasons keeping pragmatic creators of IT resources (hardware and software) from more widely using WS-Management to expose the manageability capabilities of their resources. The first one, that I will cover here, is the fear of wasting development resources (and the lack of customer demand). The second one, that I will cover in a later post, is the complexity introduced by some technical choices in WS-Management.

There is plenty of uncertainty around the status and future of WS-Management. This means that any investment in implementing the specification is at risk of having to be later thrown away. It also means that customers, while they often mention it as part of a check-list, understand that at this point WS-Management doesn’t necessarily give them the investment protection that widely-supported stable standards provide. And as such they are receptive when vendors explain that at this point there really isn’t a stable standard for manageability that goes across domains and the best they can get is support for a patchwork of established specifications like SNMP, JMX, CIM/HTTP, WMI, etc.

One source of this uncertainty about WS-Management comes from the fact that there is an equivalent standard, WSDM, that came out of OASIS. But at this point, it is pretty clear that WSDM is going nowhere. Good metrics are hard to come by, but if you compare the dates of last commit activity in the three open-source WS-Management implementations that I know of (Openwsman, Wiseman and the WS-Management module of SOA4D) to that of the Muse implementation of WSDM, you are comparing ages in hours/days to ages in months. Another way is to look at the sessions in the Web services track at the recent Management Developers Conference: six presentations around WS-Management (including an intriguing Ruby on Rails module) compared to one for WSDM. Unless your company is an IBM-only account, WSDM isn’t a useful alternative to WS-Management (and it’s not due to technical inferiority, I still prefer WSDM MUWS to WS-Management on that point but it’s largely irrelevant).

The more serious concern is that, back when it wasn’t clear that the industry would pick WS-Management over WSDM, an effort was launched to reconcile the two specifications. That effort, often refered to as the WS-Management/WSDM convergence, is private so no-one outside of the four companies involved know what is happening. The only specification that has come out at this point is a draft of WS-ResourceTransfer in summer 2006 (I don’t include WS-ResourceCatalog because even though it came out of the same group it provides features that are neither in WS-Management nor in WSDM so it is not really part of converging them). What is happening now? The convergence effort may have died silently. Or it may be on the brink of releasing a complete new set of specifications. Or it may have focused on a more modest set of enhancements to WS-Management. Even though I was in the inside until a few months ago, I am not feigning ignorance here. There is enough up in the air that I can visualize any of these options realized.

This is not encouraging to people looking to invest their meager development resources to improve manageability interfaces on their products. What if they put work in WS-Management and soon after that Microsoft, IBM, HP and Intel come out with a new set of specifications and try to convince the industry to move from WS-Management to that new set of specifications? Much safer to stay on the sidelines for now. The convergence is a source of FUD preventing adoption of WS-Management. It is, on the other hand, a lifeline for WSDM because it provides a reason for those who went with WSDM to wait and see what happens with the convergence before moving away from WSDM.

Even before leaving HP, I had come to the conclusion that it was too late for the convergence to succeed. This doesn’t imply anything about HP’s current position on the topic, which I am of course not qualified to represent. But I just noticed that the new HP BTO chief architect doesn’t seem too fond of WS-*.

Even if the convergence effort manages to deliver the specifications it promised (including an update of WS-ResourceTransfer which is currently flawed, especially its “partial put” functionality), it will be years before they get published, interop-tested, submitted and standardized. Will there be appetite for a new set of WS-* specifications at that point? Very doubtful. SOAP will be around for a long time, but the effort in the SOAP community is around using the existing set of specifications to address already-identified enterprise integration problems. The final stage in the production of any good book, article or even blog post (not that this blog is a shining example) is to pair-down the content, to remove anything that is not essential. This is the stage that the SOAP world is in, sorting through the deluge of specifications to extract and polish the productive core. New multi-spec frameworks need not apply.

If there is to emerge a new, comprehensive, framework for web-based manageability, it won’t be the WS-Management/WSDM convergence. It probably won’t use SOAP (or at least not in its WS-Addressing-infected form). It may well use RDF. But it is not in sight at this point. So for now the choice is whether to seize the opportunity to create a widely-adopted standard on the basis of WS-Management (with all its flaws) or to let the window of opportunity close, to treat WS-Management as just another manageability tool in the toolbox and go on with life. Until the stars line up in a few years and the industry can maybe take another stab at the effort. To a large extent, this is in the hands of Microsoft, IBM, HP and Intel. Ironically, the best way for those who want nothing to do with SOAP to prevent SOAP from being used too much for manageability (beyond where WS-Management is already used) is to keep pushing the convergence (which is very much SOAP based) in order to keep WS-Management contained.

3 Comments

Filed under DMTF, Everything, IT Systems Mgmt, Standards, WS-Management, WS-ResourceTransfer

December 7, 2007

Is IT management to enterprise IT what ecology is to economic development?

What happens when a society gets hold of a new territory or a new technology? It usually starts by decimating the easy preys in that territory or by running wild with the technology. Using abundant resources (food, fuel or other) with abandonment, dumping waste everywhere. Then there is a crisis directly tied to this lack of restraint. Maybe an epidemic. Or starvation from the sudden disappearance of easy-to-get food (or fuel). Lack of clean water. Landslides from deforestation. Something is done to address that crisis and its direct causes. It starts with random acts of what is not yet called ecology. And then the best practices gets more widely adopted. But another crisis appears. Other changes need to be made. Eventually people start to look beyond fighting individual fires and towards managing the environment as a whole, in a way that aligns with the desired quality of life. Models are developed to better understand relationships and predict consequences. Comprehensive environmental studies appear. People take a lifecycle approach to managing the environmental aspects of development. Processes, policies and rules get defined. And of course, companies and consultants appear to help with these tasks.

This is a (widely) simplified description of how ecology appears out of necessity in developing societies and how its development is a gating factor for sustained economic development. Of course, this is the happy view, the one where the society is able to correct its course before collapsing.

Doesn’t this sound very similar to the way IT management appeared and is developing in enterprises?

When enterprises got hold of computing as a business tool, individual departments deployed applications with little planning and coordination, just to grab the low-hanging fruits of increased productivity. Then comes the crisis, a key system goes down and no-one knows what to do. Business suffers. Some early, localized, monitoring functionality is created to fix the problem. A random act of management that addresses a tactical issue. But more problems happen, the system gets more complex than niche management tools can address. Eventually people start to look at IT management more globally, to think of it as a way to align IT with business objectives. Models are developed to better understand relationships and predict consequences. People take a lifecycle approach to managing changes to the IT environment. Best practices, processes and even rules and compliance mandates get defined. And of course, companies and consultants appear to help with these tasks.

Does this parallel reveal any opportunity for one side to learn from the other? Will you hire Greenpeace to run your data center?

Comments Off on Is IT management to enterprise IT what ecology is to economic development?

Filed under Ecology, Everything, IT Systems Mgmt, Off-topic

December 3, 2007

Virtual machine or fake machine?

In yesterday’s post I wrote a bit about the recently-announced Oracle Virtual Machine. But in the larger scheme, I have always been uncomfortable with the focus on VMWare-style virtual machines as the embodiement of “virtualization”. If a VMWare VM is a virtual machine does that mean a Java Virtual Machine (JVM) is not a virtual machine? They are pretty different. When you get a fresh JVM, the first thing you do is not to install an OS on it. To help distinguish them, I think of the VMWare style as a “fake machine” and the JVM style as an “abstract machine”. A “fake machine” behaves as similarly as possible to a physical machine and that is a critical part of its value proposition: you can run all the applications that were developed for physical machines and they shouldn’t behave any differently while at the same time you get some added benefits in terms of saving images, moving images around, more efficiently using your hardware, etc. An “abstract machine”, on the other hand, provides value by defining and implementing a level of abstraction different from that of a physical machine: developing to this level provides you with increased productivity, portability, runtime management capabilities, etc. And then, in addition to these “fake machines” and “abstract machines”, there is the virtualization approach that makes many machines appear as one, often refered to as grid computing. That’s three candidates already for carrying the “virtualization” torch. You can also add Amazon-style storage/computing services (e.g. S3 and EC2) as an even more drastic level of virtualization.

The goal here is not to collect as many buzzwords as possible within one post, but to show how all these efforts represent different ways to attack similar issues of flexibility and scalability for IT. There is plenty of overlap as well. JSRs 121 and 284, for example, can be seen as paving the way for more easily moving JVMs around, WMWare-style. Something like Oracle Coherence lives at the junction of JVM-style “abstract machines” and grid computing to deliver data services. And as always, these technologies are backed by a management infrastructure that makes them usable in the way that best serves the applications running on top of the “virtualized” (by one of the definitions above) runtime infrastructure. There is a lot more to virtualization than VMWare or OVM.

[UPDATED 2007/03/17: Toutvirtual has a nice explanation of the preponderance of “hypervisor based platforms” (what I call “fake machines” above) due to, among other things, failures of operating systems (especially Windows).]

[UPDATED 2009/5/1: For some reason this entry is attracting a lot of comment spam, so I am disabling comments. Contact me if you’d like to comment.]

1 Comment

Filed under Everything, IT Systems Mgmt, OVM, Virtualization, VMware

December 2, 2007

Oracle has joined the VM party

On the occasion of the introduction of the Oracle Virtual Machine (OVM) at Oracle World a couple of weeks ago, here are a few thoughts about virtual machines in general. As usual when talking about virtualization (see the OVF review), I come to this mainly from a systems management perspective.

Many of the commonly listed benefits of VMWare-style (I guess I can also now say OVM-style) virtualization make perfect sense. It obviously makes it easier to test on different platforms/configurations and it is a convenient (modulo disk space availability) way to distribute ready-to-use prototypes and demos. And those were, not surprisingly, the places where the technology was first used when it appeared on X86 platforms many years ago (I’ll note that the Orale VM won’t be very useful for the second application because it only runs on bare metal while in the demo scenario you usually want to be able to run it on the host OS that normally runs you laptop). And then there is the server consolidation argument (and associated hardware/power/cooling/space savings) which is where virtualization enters the data center, where it becomes relevant to Oracle, and where its relationship with IT management becomes clear. But the value goes beyond the direct benefits of server consolidation. It also lies in the additional flexibility in the management of the infrastructure and the potential for increased automation of management tasks.

Sentences that contains both the words “challenge” and “opportunity” are usually so corny they make me cringe, but I’ll have to give in this one time: virtualization is both a challenge and an opportunity for IT management. Most of today’s users of virtualization in data centers probably feel that the technology has made IT management harder for them. It introduces many new considerations, at the same time technical (e.g. performance of virtual machines on the same host are not independent), compliance-related (e.g. virtualization can create de-facto super-users) and financial (e.g. application licensing). And many management tools have not yet incorporated these new requirements, or at least not in a way that is fully integrated with the rest of the management infrastructure. But in the longer run the increased uniformity and flexibility provided by a virtualized infrastructure raise the ability to automate and optimize management tasks. We will get from a situation where virtualization is justified by statements such as “the savings from consolidation justify the increased management complexity” to a situation where the justification is “we’re doing this for the increased flexibility (through more automated management that virtualization enables), and server consolidation is icing on the cake”.

As a side note, having so many pieces of the stack (one more now with OVM) at Oracle is very interesting from a technical/architectural point of view. Not that Oracle would want to restrict itself to managing scenarios that utilize its VM, its OS, its App Server, its DB, etc. But having the whole stack in-house provides plenty of opportunity for integration and innovation in the management space. These capabilities also need to be delivered in heterogeneous environments but are a lot easier to develop and mature when you can openly collaborate with engineers in all these domains. Having done this through standards and partnerships in the past, I am pleased to be in a position to have these discussions inside the same company for a change.

1 Comment

Filed under Everything, IT Systems Mgmt, Oracle, Oracle Open World, OVM, Tech, Virtualization, VMware

November 28, 2007

CMDBf now in the hands of the DMTF

It’s now official, the CMDBf specification has been submitted to the DMTF and will be standardized there. Here is the press release and here is the specification (unchanged) republished on the DMTF site. The CMDBf working group was created a while ago at the DMTF but I didn’t report it since it wasn’t clear to me whether that was public information or not. The press release makes this clear now.

As a side note, this is one of my ongoing frustrations with the DMTF. Almost everything happens in private with no publicly-accessible URL until a press release comes out and of course lots of interesting things happen that don’t get a press release. I have heard many times that the DMTF is working on opening up the process, but I still haven’t seen much change. If this had been OASIS or W3C, the call for formation of the new working group would have been publicly accessible even before the group was created. OK, end of ranting.

As always, there isn’t much useful information to be gleaned from the text of the press release. Only that, as expected, the authors addressed the question of how this relates to CIM, since for many DMTF=CIM. So the press release proactively declares that the CMDBf work will not be limited to CIM-modeled configuration data. What this means in practice will be seen later (e.g. will there be CIM-specific extensions?).

Having seen how executive quotes for press releases get generated I hate to read too much into them, but another thing I can’t help noticing in the press release is that none of the quotes from the companies submitting the specification tout federation, but simply “integration” or “sharing”. For example: “integration and interoperability” (BMC), “share data” (CA), “sharing of information” (HP), “view, track and change information” (IBM), “exchange data” (Microsoft). This more realistic assessment of what the specification does stands in contrast to the way the DMTF presents it in the press release : “this specification provides a standard way to federate management data stored in multiple different data models”. At this point, it doesn’t really provide federation and especially not across different models.

All in all, it’s as good thing for this work to be moved to a standards organization. I may join the CMDBf group at the DMTF to track it, but I don’t plan to engage very much as this area isn’t my focus anymore now that I am at Oracle. But of course everything is linked at some level in the management field.

[UPDATE on 2007/11/30: two days after posting this message I got the monthly DMTF newsletter which touches on points I raise here. So here are the relevant links. First, Mike Baskey, DMTF Chairman, shares his view on what CMDBf means for DMTF. Second, as if to respond to my rant on the opacity of the DMTF, Josh Cohen, DMTF Vice-chairman, gives an update on process improvements. Some progress indeed, but still a far cry from opening up mailing list archives so that observers can see in real time what issues are addressed and can go back in time to understand how a specific technical decision was made and what were the considerations.]

Comments Off on CMDBf now in the hands of the DMTF

Filed under CMDB, CMDB Federation, CMDBf, DMTF, Everything, IT Systems Mgmt, ITIL, Specs, Standards

November 6, 2007

Illustrative algorithm for CMDBf 1.0 Query operation

When I posted an algorithm for the server side implementation of a CMDBf Query call for version 0.95 of the specification, the interoperability testing session based on that version was over and I was pretty sure no-one but those of us who participated in that session would write an implementation of 0.95. But I published the algorithm anyway since I thought it was helpful to anyone who wanted to understand the specification in depth, even if they were not implementing it. Now that 1.o is out, there is a much higher probability of people implementing the specification, so I figured it would be worth updating the algorithm to take into account the changes from 0.95 to 1.0. So here it is.

One caveat. This algorithm assumes that the query request does not make use of the xpathExpression element because, as I have explained in my review of CMDBf 1.0, I don’t think interoperability is achievable on this feature in the current state of the specification.

As a note of caution, the previous version of the algorithm was backed by my implementation of CMDBf 0.95 for the interoperability testing, so I felt pretty confident about it. For this version of the algorithm I have not written a corresponding implementation and I have not done interoperability testing with anyone, it’s just based on my reading of the specification. The handling of depthLimit in particular is a little tricky and needs to be validated by implementation (what with creating a bunch of dummy item and relationship templates with temporary names and later going back to the original template names), please let me know if you find it flawed.

And, as previously, this is in no way an optimal implementation strategy. It is the most direct and obvious set of steps that I can come up with to implement the Query call in a way that exactly conforms to the specification. There are lots of ways to make this go faster, such as the ones I mentioned in a previous post (e.g. breaking out of loops once an instance has been removed, or not recalculating L1 and L2 over and over again for relationships in the same working set that share a source/target) plus new ones such as being smarter than my brute-force approach to handling depthLimit (in step 2).

All this said, here is the algorithm:

1) for each itemTemplate, calculate the set of all items (including relationships since they are a subclass of item) that obey the instanceIdConstraint and recordConstraint elements in the template (if present). Call this the working set for the itemTemplate.
2) for each relationshipTemplate RT that has a depthLimit element:

2.1) for i ranging from 1 to the value of maxIntermediateItems for RT:

2.1.1) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s sourceTemplate, except that it has a new, unique temporary id (keep a record linking that new id to the id of the original source itemTemplate).
2.1.2) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s targetTemplate, except that it has a new, unique, temporary id (keep a record linking that new id to the id of the original target itemTemplate).
2.1.3) for j ranging 1 from i:

2.1.3.1) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s intermediateItemTemplate, except that it has a new, unique, temporary id (keep a record linking that new id to the id of the original intermediary itemTemplate).
2.1.3.2) create a relationshipTemplate that is an exact copy of RT, except that its source is the itemTemplate created in the previous iteration of the current loop (or the itemTemplate created in step 2.1.1 if j=1), its target is the itemTemplate created in the previous step and it has a new, unique, temporary id (keep a record linking that new id to RT’s id).

2.1.4) create a relationshipTemplate that is an exact copy of RT, except that its source is the last itemTemplate created in the 2.1.3 loop, its target is the itemTemplate created in 2.1.2 and it has a new, unique, temporary id (keep a record linking that new id to RT’s id).

3) for each relationshipTemplate calculate the set of all relationships that obey the instanceIdConstraint and recordConstraint elements in the template (if present). Call this the working set for the relationshipTemplate.
4) set need_to_loop = true
5) while (need_to_loop == true)

5.1) set need_to_loop = false
5.2) for each relationshipTemplate RT

5.2.1) let ITsource be the itemTemplate that is referenced as sourceTemplate by RT. Calculate the set of all items (including relationships since they are a subclass of item) that obey at least one of the instanceIdConstraint elements in ITsource (assuming there is at least one such element) and all the recordConstraint elements in ITsource. Call this the working set for ITsource.
5.2.2) let ITtarget be the itemTemplate that is referenced as targetTemplate by RT. Calculate the set of all items (including relationships since they are a subclass of item) that obey at least one of the instanceIdConstraint elements in ITtarget (assuming there is at least one such element) and all the recordConstraint elements in ITtarget. Call this the working set for ITtarget.
5.2.3) for each relationship R in the working set for RT

5.2.3.1) if the source of R is not in the working set for ITsource, then remove R from the RT working set
5.2.3.2) if the target of R is not in the working set for ITtarget, then remove R from the RT working set
5.2.3.3) if RT has a source/@minimum or a source/@maximum attribute

5.2.3.3.1) find the list L1 of all relationships in the working set for RT that have the same source as R
5.2.3.3.2) if RT has source/@minimum and the cardinality of L1 is less than this minimum then remove all relationships in L1 from the RT working set
5.2.3.3.3) if RT has source/@maximum and the cardinality of L1 is more than this maximum then remove all relationships in L1 from the RT working set

5.2.3.4) if RT has a target/@minimum or a target/@maximum attribute

5.2.3.4.1) find the list L2 of all relationships in the working set for RT that have the same target as R
5.2.3.4.2) if RT has target/@minimum and the cardinality of L2 is less than this minimum then remove all relationships in L2 from the RT working set
5.2.3.4.3) if RT has target/@maximum and the cardinality of L2 is more than this maximum then remove all relationships in L2 from the RT working set

5.3) for each itemTemplate IT:

5.3.1) let sourceRTset be the set of all relationshipTemplates that references IT as its sourceTemplate
5.3.2) let targetRTset be the set of all relationshipTemplates that references IT as its targetTemplate
5.3.3) for each item I in the IT working set

5.3.3.1) for each relationshipTemplate sourceRT in sourceRTset, if there is no relationship in the working set for sourceRT that uses I as its source, remove I from the IT working set and set need_to_loop to true
5.3.3.2) for each relationshipTemplate targetRT in targetRTset, if there is no relationship in the working set for targetRT that uses I as its source, remove I from the IT working set and set need_to_loop to true

6) process the eventual contentSelector elements and/or the @suppressFromResult attributes on the templates that have matching items/relationships in the response to remove or pair down items and relationships as requested
7) package the resulting items and relationships in a way that conforms to the CMDBf response message format (including putting each item in the <nodes> element with the appropriate @templateId attribute and putting each relationship in the <edges> element with the appropriate @templateId).
8) replace all the temporary template ids (from step 2) that appear in templateId attributes in the response with the original ids of the items and template based on the records that were kept in step 2.

Just to clarify things, what I do in step 2 is simply make explicit all the itemTemplates and relationshipTemplates that are made implicit by the depthLimit element, so that we can provide with a simpler algorithm after that assumes that all relationshipTemplate correspond to direct relationships (no intermediary). And in step 8 I hide the fact that this took place.

[UPDATED 2009/5/1: For some reason this entry is attracting a lot of comment spam, so I am disabling comments. Contact me if you’d like to comment.]

8 Comments

Filed under CMDB, CMDB Federation, CMDBf, Graph query, IT Systems Mgmt, Pseudo-algorithm, Query, Specs, Standards, Tech

November 1, 2007

The Oslo accords (presumably between composite application modeling and systems management)?

Microsoft introduced an umbrella project called Oslo at their SOA and Business Process conference this week. There is very little information available but it seems to have two main components: improving the ability of the Microsoft platform to support SOA-style distributed applications and improving the use of models to develop and manage applications. At first sight there isn’t anything new. The SOA talk is similar to any number of “why SOA” presentations available from dozens of companies. And the modeling aspect is the same story that Microsoft has been pitching with DSI for years. The real news is that the two stories are being linked (at least at the marketing level, which is a starting point) and that the application development people have taken over the application modeling baton from the System Center group.

Over the last few years, I worked with people from System Center on different standards related to DSI, including SML which they see as the heart of the modeling effort. One of the things that kept me skeptical when hearing the DSI pitch, was to see the System Center team making announcement and promises about how SML would be central to the development experience in Visual Studio. I am pretty sure I know who’s the gorilla and who’s the chimp at Microsoft between Visual Studio / .Net Framework on the one hand and System Center on the other. The application model is too central to the developer experience for the Visual Studio group not to own it. It looks like it’s now happening and it’s a good thing.

The only content I could find on Oslo that’s not PR fluff is a report from Directions on Microsoft which mostly talks about incremental improvements to BizTalk. Towards the end, there is a small section about a “repository” that will “provide centralized storage of composite application components”. At that point I can’t help remembering the blog post from David Chappell about why it wouldn’t make sense for Microsoft to support SCA. Through comments in his post as well as a blog post of my own, I followed-up with the assertion that the application component model also plays a very important role for management. And at the risk of sounding self-congratulatory, the Oslo announcement seems vindicate that view. I see that David was a speaker at the Microsoft conference where Oslo was announced and he has very good insights into both the application developement and the systems management efforts at Microsoft. So hopefully he’ll soon have a white paper or a blog entry out to share some insights.

If you’re wondering what this means for the technical work that has been going on under the DSI umbrella so far, you can only read the tea leaves. It could be that the application development people adopted the whole SML/CML technology stack as promoted by their System Center colleagues and are going to use it as is. Or on the other extreme, it could be a complete reset that leads to the creation of a component model that is much less general and much more application-centric. Of course, no matter which one happens (or something in the middle), it will be presented as a perfectly smooth and controlled evolution of the DSI vision (get ready for some nice spin at MMS2008). If you are adopting SML because you expect Microsoft to base its application component model on it, you might want to wait a bit until more details emerge about Oslo. For example, after calling XSD “a schema language that attempts to be a floor wax, dessert topping, and personal lubricant all at the same time” you have to wonder whether Don Box would advocate to use SML (80% of which is XSD) as the most effective metamodel for an application component model…

Let’s end with this quote from the Directions on Microsoft report on Oslo, regarding application integration: “SAP and Oracle are better positioned in that regard, and so their customers will want to investigate these vendors’ composite application platforms along side Microsoft’s”. Can’t disagree with that. A good place to start this investigation would be the upcoming Oracle Open World.

3 Comments

Filed under IT Systems Mgmt, Microsoft, Oslo, SCA, SML, Tech

October 28, 2007

Review of the CMDBf specification version 1.0

Having read the recently released CMDBf 1.0 specification over the weekend, I see several improvements since 0.95, including:

the introduction of depthLimit
the lastModified metadata element
the ability to specify more than one instanceId in a template
the ability to advertise what parts of the specification you implement
the definition of faults

But while 1.0 is more complete than 0.95, I think it makes it harder to achieve interoperability. Here are the main friction points for interop:

New role for XPath

The xpathExpression element (which replaces xpath1Selector) changes in two very important ways. First, rather than being limited to XPath 1.0, it now also allows XPath 2.0. Support for this is a lot harder to achieve for people who don’t use XML as the backend format for their data. Considering the current state of adoption of XPath 2.0 and the low level of XML complexity exposed by most CMDB models, I don’t think it was opportune to bring this into CMDBf yet. And my guess is that most implementations will stay away from this. But there is a second change, less obvious but even more problematic. XPath is not just another constraint mechanism for a CMDBf template anymore, one that returns a boolean result indicating whether the instance meets the constraint or not, as it used to be in 0.95. It is now an alternative selection and filtering mechanism that lives in parallel to all the other elements in a template (and can’t mix with them). Overall, I think this change goes too far in the direction of turning a shared agreement to exchange data in XML into an assumption that the internal data models are all based on XML. And the killer with regards to interoperability is that the specification says nothing about how the resulting node sets are serialized in the response. There may be a serialization for the XPath 2.0 model, but there is no such thing for XPath 1.0 and I don’t see in the current state of the specification how two implementations have any chance to interoperate when using this feature.

Introduction of linkDepth

As I mentioned earlier, linkDepth is a very useful addition (even though it pales in comparison to the inferencing capabilities that could have been derived from basing CMDBf on RDF). But it is also a complicated feature. The intermediateItemTemplate attribute is a good re-use of the existing plumbing, but it needs at least a detailed example. I trust that the group will generate one once they’ve caught their breath from putting out the specification.

Service capability metadata

There is a new section (#6) to provide ways to describe what CMDBf features an implementation supports. But it is a very granular representation. Basically, for every feature you can describe if you support it or not. So someone may describe that they support everything inside propertyValue, except for the “like” operator. And someone else might support all the operators but not the caseSensitive modifier. That might be ok for human consumptions, but automated scenarios rely on pre-programmed queries and that is made very hard by all the possible combinations of options. What we need is a few well-defined profiles that people implement fully. Starting of course with a profile that rules out xpathExpression.

Record metadata

This new version introduces metadata on records. While recordId and lastModified are probably well understood and interoperably usable I am a bit more dubious about whether baselineId and snapshotId are going to be interoperable across vendors based on their limited description in the specification. The nice thing is that this metadata can not only be returned but also searched on. Well, at least that’s the intent. But this goes through the recordMetadata attribute on propertyValue which, while present in the pseudo-schema, is missing in the XSD…

The contentSelector element

This new element is more flexible that the propertySubsetDirective element that it replaces. In addition to specifying what properties you want returned it also allows you to specify that you only want certain record types and/or that you only want the record(s) that were used to satisfy constraints in the template. Those are nice additions, but the way the second part is implemented (through the use of the matchedRecords attribute) seems to assume that only one record in the instance was used to match all the constraints in the template. This is not necessarily the case, an instance can be selected by having different records match the different constraints in the template as long as it has at least one matching record per constraint (line 765 says “the item satisfies all the constraints”, not “a record of the item satisfies all the constraints” and you can also see this in the example in section 4.2 where the records mentioned on lines 637 and 639 don’t have to be the same). So do you return all records that have a role in matching the template, or only those (if there is any) that matches all the constraints on their own as the text seems to imply? And if several record combinations inside an instance can be used to match the constraints in a template, do I return all of them or can I just pick any subset that matches? Also, how can I say that I want all records that established the template match, independently of their type? There doesn’t seem to be a way to do this, or is it by putting a contentSelector element with no child element and the matchedRecords attribute set to false? There won’t be much interoperability on this feature until all this is clarified.

Relationships as items

A major change between 0.95 and 1.0 is that now a relationship can match an itemTemplate. For example, if you ask for all items that were modified during the last 24 hours you will get all the items and all the relationships that meet that criteria while in the previous version you’d have to explicitly request the relationships with a relationshipTemplate if you wanted to get them too). There is a good case to be made for either view and the one that works best largely depends on your backend implementation technology (RDF, objects, SQL, CIM…). But the important thing is for the spec to be clear and on this point I think the change wasn’t made explicit enough in the query section of the specification. If Van hadn’t called my attention to this on his blog, I would have missed this important change.

Security boilerplate

There is a person at IBM (probably located in a well-stoked underground bunker in upstate NY) who has instilled the fear of god in all IBM employees (at least all those who author publicly available specifications) and forces them to include a boilerplate “security considerations” section everywhere. I have co-authored several documents with IBM employees and it never fails, even thought it doesn’t add anything useful to the specification. You should see the look of fear on the face of the IBM employees when someone else suggests doing without it. We somehow managed to sneak one such slimmer specification past the IBMers with CMDBf 0.95 but I see that this has been “corrected” in 1.0. I hope that whatever painful punishment Scott, Jacob, Andrew and Mark (or their families and pets) were subjected to in the process by the IBM security ogre wasn’t too cruel. Sure, this doesn’t really impact interoperability, but now that I don’t work for a company that makes money from ink anymore, I have even less patience for this bloating.

OK, that’s enough back seat driving for now. Hopefully the standards group that will take over the specification will address all these questions. In the context of the entire specification, these are pretty small issues and mostly easy to fix. And the CMDBf group can go on to address the hard issues of federation (including security-related issues that abound in this field if one really wants to tackle them). The current specification is a useful graph-oriented query language that is a good match for CMDB data. But it’s really just a query language (plus a simple registration system).

[UPDATE: while updating the CMDBf query algorithm, I noticed another small error: maxIntermediateItems is an attribute in the pseudo-schema but an element in the schema. Something else to fix in the next version.]

3 Comments

Filed under CMDB, CMDB Federation, CMDBf, Everything, Graph query, IT Systems Mgmt, ITIL, Query, Specs, Standards, Tech

October 25, 2007

CMDBf 1.0 specification released

The CMDBf committee has just released version 1.0 of the specification. Van Wiles has an overview of the changes between 0.95 and 1.0. I left HP soon after 0.95 was released and that’s when my participation in CMDBf ended, so Van’s summary is very useful to me. The changes he lists are not surprising and some of them already existed in draft form before 0.95 publication. I need to spend some intimate time with the specification to review the changes to the template mechanism in more details. Some of the changes have the potential to make the specification quite a bit harder to implement. This is especially the case for the introduction of “depthLimit” (but it’s probably a needed feature anyway). And the fact that relationships can now match item selectors will make things either easier or harder to implement, depending on your implementation choice (e.g. straight-to-SQL/XML or through an OO or RDF model). Congrats to the group. We should soon hear about submission for standardization.

Comments Off on CMDBf 1.0 specification released

Filed under CMDB Federation, CMDBf, Everything, IT Systems Mgmt, Query, Specs, Standards, Tech

October 1, 2007

SCA is not just for code portability

(updated on 2007/10/4, see bottom of the article)

David Chappell (not the same person as the Oracle-employed Dave Chappell from my previous post) has a blog entry explaining why there would be little value if Microsoft implemented SCA. The entry is reasonable but, like this follow-up by Stephan Tilkov, it focuses on clarifying the difference between portability (for which SCA helps) and interoperability (for which SCA doesn’t help very much). Seeing it from the IT management point of view, I see another advantage to SCA: it’s a machine readable description of the logic of the composite application, at a useful level of granularity for application and service management. This is something I can use in my application infrastructure to better understand relationships and dependencies. It brings the concepts of the application world to a higher level of abstraction (than servlets, beans, rows etc), one in which I can more realistically automate tasks such as policy propagation, fail-over automation, impact analysis, etc.

As a result, even if this was an Oracle-only technology, I would still be encouraging Greg and others to build it in the Oracle app server so that I can better managed applications written on that stack. And I would still encourage the Oracle Fusion applications to take advantage of it, for the same reason.

In that perspective, going back to Dave Chappell’s question, would there be value if Microsoft implemented SCA? I think so. It would make it a lot easier for me, and all the management vendors, to efficiently manage composite applications that have components running on both Microsoft and Oracle, for example. I believe Microsoft will need a component model for composite applications and I am sure Don Box has his ideas on this (he’s not yet ready to share his opinion on Dave’s question as you can see). I know of the SML-based work that is being driven by the System Center guys at Microsoft and they see SML as playing that role across applications and infrastructure. I don’t know how much they’ve convinced Don and others that this is the right way.

From an IT management perspective, portability of code doesn’t buy me very much. Portability of my ability to introspect composite applications and consume their metadata independently of the stack they are built on, on the other hand, is of great value. Otherwise we’ll only be able to build automated and optimized application and service management software for the Oracle stack. Which, I guess, would not be a bad first step…

[UPDATE on 2007/10/4] If this topic is of interest to you, you might want to go back to some of the links above to read the comments. David Chappell and I had a little back-and-forth in the comments section of his post, and so with Don Box in his post. In addition, Hartmut Wilms at InfoQ provides his summary of the discussion.]

3 Comments

Filed under Everything, IT Systems Mgmt, Portability, SCA

September 26, 2007

Grid-enabled SOA article

Dave Chappell and David Berry have recently published an article in SOA Magazine titled “The Next-Generation, Grid-Enabled Service-Oriented Architecture”. I had unexpectedly gotten a quick overview of this work a few weeks ago, when I ran into Dave Chappell at the Oracle gym (since I was coming out of an early morning swim it took Dave a couple of seconds to recognize me, as I walked through the weight room leaving a trail of water behind me). Even if you are more interested in systems management than middleware, this article is worth your reading time because it describes a class of problems (or rather, opportunities) that cut across middleware and IT management. Namely, providing the best environment for scalable, reliable and flexible SOA applications. In other words, making the theoretical promises of SOA practically achievable in real scenarios. The article mentions “self-healing management and SLA enforcement” and it implies lots of capabilities to automatically provision, configure and manage the underlying elements of the Grid as well the SOA applications that make use of it. Those are the capabilities that I am now working on as part of the Oracle Enterprise Manager team. And the beauty of doing this at Oracle, is that we can work on this hand in hand with people like Dave to make sure that we don’t create artificial barriers between middleware and systems management.

1 Comment

Filed under Articles, Everything, Grid, Grid-enabled SOA, IT Systems Mgmt, Tech

September 16, 2007

A review of OVF from a systems management perspective

I finally took a look at OVF, the virtual machine distribution specification that was recently submitted to DMTF. The document is authored by VMware and XenSource, but they are joined in the submission to DMTF by some other biggies, namely Microsoft, HP, IBM and Dell.

Overall, the specification does a good job of going after the low-hanging fruits of VM distribution/portability. And the white paper is very good. I wish I could say that all the specifications I have been associated with came accompanied by such a clear description of what they are about.

I am not a virtualization, operating system or hardware expert. I am mostly looking at this specification from the systems management perspective. More specifically I see virtualization and standardization as two of the many threads that create a great opportunity for increased automation of IT management and more focus on the application rather than the infrastructure (which is part of why I am now at Oracle). Since OVF falls in both the “virtualization” and “standardization” buckets, it got my attention. And the stated goal of the specification (“facilitate the automated, secure management not only of virtual machines but the appliance as a functional unit”, see section 3.1) seems to fit very well with this perspective.

On the other hand, the authors explicitly state that in the first version of the specification they are addressing the package/distribution stage and the deployment stage, not the earlier stage (development) or the later ones (management and retirement). This sidesteps many of the harder issues, which is part of why I write that the specification goes after the low-hanging fruits (nothing wrong with starting that way BTW).

The other reason for the “low hanging fruit” statement is that OVF is just a wrapper around proprietary virtual disk formats. It is not a common virtual disk format. I’ve read in several news reports that this specification provides portability across VM platforms. It’s sad but almost expected that the IT press would get this important nuance wrong, it’s more disappointing when analysts (who should know better) do, as for example the Burton Group which writes in its analysis “so when OVF is supported on Xen and VMware virtualization platforms for example, a VM packaged on a VMware hypervisor can run on a Xen hypervisor, and vice-versa”. That’s only if someone at some point in the chain translates from the Xen virtual disk format to the VMware one. OVF will provide deployment metadata and will allow you to package both virtual disks in a TAR if you so desire, but it will not do the translation for you. And the OVF authors are pretty up front about this (for example, the white paper states that “the act of packaging a virtual machine into an OVF package does not guarantee universal portability or install-ability across all hypervisors”). On a side note, this reminds me a bit of how the Sun/Microsoft Web SSO MEX and Web SSO Interop Profile specifications were supposed to bridge Passport with WS-Federation which was a huge overstatement. Except that in that case, the vendors were encouraging the misconception (which the IT press happily picked up) while in the OVF case it seems like the vendors are upfront about the limitations.

There is nothing rocket-science about OVF and even as a non-virtualization expert it makes sense to me. I was very intrigued by the promise that the specification “directly supports the configuration of multi-tier applications and the composition of virtual machines to deliver composed services” but this turns out to be a bit of an overstatement. Basically, you can distribute the VMs across networks by specifying a network name for each VM. I can easily understand the simple case, where all the VMs are on the same network and talking to one another. But there is no way (that I can see) to specify the network topology that joins different networks together, e.g. saying that there is a firewall between networks “blue” and “red” that only allows traffic on port 80). So why would I create an OVF file that composes several virtual machines if they are going to be deployed on networks that have no relationships to one another? I guess the one use case I can think of would be if one of the virtual machines was assigned to two networks and acted as a gateway/firewall between them. But that’s not a very common and scalable way to run your networks. There is a reason why Cisco sells $30 billions of networking gear every year. So what’s the point of this lightweight distributed deployment? Is it just for that use case where the network gear is also virtualized, in the expectation of future progress in that domain? Is this just a common anchor point to be later extended with more advanced network topology descriptions? This looks to me like an attempt to pick a low-hanging fruit that wasn’t there.

Departing from usual practice, this submission doesn’t seem to come with any license grant, which must have greatly facilitated its release and the recruitment of supporters for the submission. But it should be a red flag for adopters. It’s worth keeping track of its IP status as the work progresses. Unless things have changed recently, DMTF’s IP policy is pretty weak so the fact that works happens there doesn’t guarantee much protection per se to the adopters. Interestingly, there are two sections (6.2 about the virtual disk format and 11.3 about the communication between the guest software and the deployment platform) where the choice of words suggests the intervention of patent lawyers: phrases like “unencumbered specification” (presumably unencumbered with licensing requirements) and “someone skilled in the art”. Which is not surprising since this is the part where the VMWare-specific, Xen-specific or Microsoft-specific specifications would plug in.

Speaking of lawyers, the section that allows the EULA to be shipped with the virtual appliance is very simplistic. It’s just a human-readable piece of text in the OVF file. The specification somewhat naively mentions that “if unattended installs are allowed, all embedded license sections are implicitly accepted”. Great, thanks, enterprises love to implicitly accept licensing terms. I would hope that the next version will provide, at least, a way to have a URI to identify the EULA so that I can maintain a list of pre-approved EULAs for which unattended deployment is possible. Automation of IT management is supposed to makes things faster and cheaper. Having a busy and expensive lawyer read a EULA as part of my deployment process goes against both objectives.

It’s nice of the authors to do the work of formatting the specification using the DMTF-approved DSPxxxx format before submitting to the organization. But using a targetnamespace in the dmtf.org domain when the specification is just a submission seems pretty tacky to me, unless they got a green light from the DMTF ahead of time. Also, it looks a little crass on the part of VMware to wrap the specification inside their corporate white paper template (cover page and back page) if this is a joint publication. See the links at http://www.vmware.com/appliances/learn/ovf.html. Even though for all I know VMware might have done most of the actual work. That’s why the links that I used to the white paper and the specification are those at XenSource, which offers the plain version. But then again, this specification is pretty much a wrapper around a virtual disk file, so graphically wrapping it may have seemed appropriate…

OK, now for some XML nitpicking.

I am not a fan of leaving elementformdefault set to “unqualified” but it’s their right to do so. But then they qualify all the attributes in the specification examples. That looks a little awkward to me (I tend to do the opposite and qualify the elements but not the attributes) and, more importantly, it violates the schema in appendix since the schema leaves attributeFormDefault to its default value (unqualified). I would rather run a validation before makings this accusation, but where are the stand-alone XSD files? The white paper states that “it is the intention of the authors to ensure that the first version of the specification is implemented in their products, and so the vendors of virtual appliances and other ISV enablement, can develop to this version of the specification” but do you really expect us to copy/paste from PDF and then manually remove the line numbers and header/footer content that comes along? Sorry, I have better things to do (like whine about it on this blog) so I haven’t run the validation to verify that the examples are indeed in violation. But that’s at least how they look to me.

I also have a problem with the Section and Content elements that are just shells defined by the value of their xsi:type attribute. The authors claim it’s for extensibility (“the use of xsi:type is a core part of making the OVF extensible, since additional type definitions for sections can be added”) but there are better ways to do extensibility in XML (remember, that’s what the X stands for). It would be better to define an element per type (disk, network…). They could possibly be based on the same generic type in XSD. And this way you get more syntactic flexibility and you get the option to have sub-types of sub-types rather than a flat list. Interestingly, there is a comment inside the XSD that defines the Section type that reads “the base class for a section. Subclassing this is the most common form of extensibility”. That’s the right approach, but somehow it got dropped at some point.

Finally, the specification seems to have been formated based on WS-Management (which is the first specification that mixed the traditional WS-spec conventions with the DMTF DSPxxxx format), which may explain why WS-Management is listed as a reference at the end even though it is not used anywhere in the specification. That’s fine but it shows in a few places where more editing is needed. For example requirement R1.5-1 states that “conformant services of this specification MUST use this XML namespace Universal Resource Identifier (URI): http://schemas.dmtf.org/ovf”. I know what a conformant service is for WS-Management but I don’t know what it is for this specification. Also, the namespace that this requirement uses is actually not defined or used by this specification, so this requirement is pretty meaningless. The table of namespaces that follows just after is missing some namespaces. For example, the prefix “xsi” is used on line 457 (xsi:any and xsi:AnyAttribute) and I want to say it’s the wrong one as xsi is usually assigned to “http://www.w3.org/2001/XMLSchema-instance” and not “http://www.w3.org/2001/XMLSchema” but since the prefix is not in the table I guess it’s anyone’s guess (and BTW, it’s “anyAttribute”, not “AnyAttribute”).

By this point I may sound like I don’t like the specification. Not at all. I still stand with what I wrote in the second paragraph. It’s a good specification and the subset of problems that it addresses is a useful subset. There are a few things to fix in the current content and several more specifications to write to complement it, but it’s a very good first step and I am glad to see VMware and XenSource collaborating on this. Microsoft is nominally in support at this point, but it remains to be seen to what extent. I haven’t seen them in the past very interested in standards effort that they are not driving and so far this doesn’t appear to be something they are driving.

6 Comments

Filed under DMTF, Everything, IT Systems Mgmt, OVF, Specs, Standards, Tech, Virtualization, VMware, XenSource