William Vambenepe's blog

Registering a domain name: $10 per year
Hosting it with all the features you may need: $80 per year
Controlling your on-line life: priceless

To be frank, the main reason that I do not use Facebook or MySpace is that I am not very social to start with. But, believe it or not, I have a few friends and family member with whom I share photos and personal stories. Not to mention this blog for different kinds of friends and different kinds of stories (you are missing out on the cute toddler photos).

Rather than doing so on Facebook, MySpace, BlogSpot, Flickr, Picasa or whatever the Microsoft copies of these sites are, I maintain a couple of blogs and on-line photo albums on vambenepe.com. They all provide user access control and RSS-based syndication so no-one has to come to vambenepe.com just to check on them. No annoying advertising, no selling out of privacy and no risk of being jerked around by bait-and-switch (or simply directionless) business strategies (”in order to serve you better, we have decided that you will no longer be able to download the high-resolution version of your photos, but you can use them to print with our approved print-by-mail partners”). Have you noticed how people usually do not say “I use Facebook” but rather “I am on Facebook” as if riding a mechanical bull?

The interesting thing is that it doesn’t take a computer genius to set things up in such a way. I use Dreamhost and it, like similar hosting providers, gives you all you need. From the super-easy (e.g. they run WordPress for you) to the slightly more personal (they provide a one-click install of your own WordPress instance backed by your own database) to the do-it-yourself (they give you a PHP or RoR environment to create/deploy whatever app you want). Sure you can further upgrade to a dedicated server if you want to install a servlet container or a CodeGears environment, but my point is that you don’t need to come anywhere near this to own and run your own on-line life. You never need to see a Unix shell, unless you want to.

This is not replacing Facebook lock-in with Dreamhost lock-in. We are talking about an open-source application (WordPress) backed by a MySQL database. I can move it to any other hosting provider. And of course it’s not just blogging (WordPress) but also wiki (MediaWiki), forum (phpBB), etc.

Not that every shinny new on-line service can be replaced with a self-hosted application. You may have to wait a bit. For example, there is more to Facebook than a blog plus photo hosting. But guess what. Sounds like Bob Bickel is on the case. I very much hope that Bob and the ex-Bluestone gang isn’t just going to give us a “Facebook in a box” but also something more innovative, that makes it easy for people to run and own their side of a Facebook-like presence, with the ability to connect with other implementations for the social interactions.

We have always been able to run our own web applications, but it used to be a lot of work. My college nights were soothed by the hum of an always-running Linux server (actually a desktop used as a server) under my desk on which I ran my own SMTP server and HTTPd. My daughter’s “soothing ocean waves” baby toy sounds just the same. There were no turnkey web apps available at the time. I wrote and ran my own Web-based calendar management application in Python. When I left campus, I could have bought some co-locating service but it was a hassle and not cheap, so I didn’t bother [*].

I have a lot less time (and Linux administration skills) now than when I left university, so how come it is now attractive for me to run my own web apps again? What changed in the environment?

The main driver is the rise of the LAMP stack and especially PHP. For all the flaws of the platform and the ugliness of the code, PHP has sparked a huge ecosystem. Not just in terms of developers but also of administrators: most hosting providers are now very comfortable offering and managing PHP services.

The other driver is the rise of virtualization. Amazon hosts Xen images for you. But it’s not just the hypervisor version of virtualization. My Dreamhost server, for example, is not a Xen or VMWare virtual machine. It’s just a regular server that I share with other users but Dreamhost has created an environment that provides enough isolation from other users to meet my needs as an individual. The poor man’s virtualization if you will. Good enough.

These two trends (PHP and virtualization) have allowed Dreamhost and others to create an easy-to-use environment in which people can run and deploy web applications. And it becomes easier every day for someone to compete with Dreamhost on this. Their value to me is not in the hardware they run. It’s in environment they provide that prevents me from having to do low-level LAMP administration that I don’t have time for. Someone could create such an environment and run it on top of Amazon’s utility computing offering. Which is why I am convinced that such environments will be around for the foreseeable future, Dreamhost or no Dreamhost. Running your own web applications won’t be just for geeks anymore, just like using a GPS is not just for the geeks anymore.

Of course this is not a panacea and it won’t allow you to capture all aspects of your on-line life. You can’t host your eBay ratings. You can’t host your Amazon rank as a reviewer. It takes more than just technology to break free, but technology has underpinned many business changes before. In addition to the rise of LAMP and virtualization already mentioned, I am watching with interest the different efforts around data portability: dataportability.org, OpenID, OpenSocial, Facebook API… Except for OpenID, these efforts are driven by Web service providers hoping to canalize the demand for integration. But if they are successful, they should give rise to open source applications you can host on your own to enjoy these services without the lock-in. One should also watch tools like WSO2’s Mashup Server and JackBe Presto for their potential to rescue captive data and exploit freed data. On the “social networks” side, the RDF community has been buzing recently with news that Google is now indexing FOAF documents and exposing the content through its OpenSocial interface.

Bottom line, when you are offered to create a page, account or URL that will represent you or your data, take a second to ask yourself what it would take to do the same thing under your domain name. You don’t need to be a survivalist freak hiding in a mountain cabin in Montana (”it’s been eight years now, I wonder if they’ve started to rebuild cities after the Y2K apocalypse…”) to see value in more self-reliance on the web, especially when it can be easily achieved.

Yes, there is a connection between this post and the topic of this blog, IT management. It will be revealed in the next post (note to self: work on your cliffhangers).

[*] Some of my graduating colleagues took their machines to the dorm basement and plugged them into a switch there. Those Linux Slackware machines had amazing uptimes of months and years. Their demise didn’t come from bugs, hacking or component failures (even when cats made their litter inside a running computer with an open case) but the fire marshal, and only after a couple of years (the network admins had agreed to turn a blind eye).

[UPDATED 2008/7/7: Oh, yeah, another reason to run your own apps is that you won't end up threatened with jail time for violating the terms of service. You can still end up in trouble if you misbehave, but they'll have to charge you with something more real, not a whatever-sticks approach.]

I see that WSO2 has just released version 1.0 of their Mashup Server. Congratulations to Jonathan and the rest of the team. I haven’t played with the earlier betas of the Mashup Server but I have read enough about it to be interested. Now that it’s been released, it might be a good time to invest a few hours to look into it (I just downloaded it and I filled a small documentation bug already). I know (and like) many of the WSO2 guys (Jonathan, but also Sanjiva and Glen) from the early days of the W3C WSDL working group. Plus, you have to give credit to a company that offers visibility on its web site not just to its board and management team but also to its engineers.

But the Mashup Server is not interesting to me just because I know some of its authors. There are tow more important reasons. One is that it is the integration product in WSO2’s portfolio that is the most different in its approach from the many integration products in Oracle Fusion Middleware. We want Oracle Enterprise Manager to do an outstanding job at managing Oracle Fusion Middleware, but we also want it to manage other integrations approaches as well (we manage Tomcat for example). At this point there is of course no market demand for managing WSO2’s Mashup Server, but from an architectural perspective it’s a good alternative to keep in mind along with the BPEL, ESB, ODI, etc that are already in heavy use. I am always interested in perspectives that help make sure that the most abstract application/service management concepts remain suitably abstracted, so learning a bit about the Mashup Server can’t hurt. I’ll know more once I’ve looked at it, but my impression is that the Mashup Server is somewhere between BPEL and Ruby on Rails (or TurboGears) in terms of declarativity and introspectability (yes I like to make up words) for management purposes.

This may well be sweet spot and it’s my second reason for being interested in the Mashup Server. I am always interested in tools that help with quick prototyping and the best tool is different for each job. The Mashup Server is pretty unique and I can imagine it being a nice tool for some management integration prototypes once the participating services have been suitably XML-ized (something that that Oracle Fusion Middleware makes easy).

Interestingly, the release of this JavaScript-based platform comes on the same day that Joe Gregorio declares JavaScript to be the new SmallTalk.

Over the last few months, I have seen two “top 10″ lists with almost the same title and nearly zero overlap in content. One is Network World’s “10 virtualization companies to watch” published in August 2007. The other is CIO’s “10 Virtualization Vendors to Watch in 2008″ published three months later. To be precise, there is only one company present in both lists, Marathon Technologies. Congratulations to them (note to self: hire their PR firm when I start my own company). Things are happening quickly in that field, but I doubt the landscape changed drastically in these three months (even though the announcement of Oracle’s Virtual Machine product came during that period). So what is this discrepancy telling us?

If anything, this is a sign of the immaturity of the emerging ecosystem around virtualization technologies. That being said, it could well be that all this really reflects is the superficiality of these “top 10″ lists and the fact that they measure PR efforts more than any market/technology fact (note to self: try to become less cynical in 2008) (note to self: actually, don’t).

So let’s not read too much into the discrepancy. Less striking but more interesting is the fact that these lists are focused on management tools rather than hypervisors. It is as if the competitive landscape for hypervisors was already defined. And, as shouldn’t be a surprise, it is defined in a way that closely mirrors the operating system landscape, with Xen as Linux (the various Xen-based offerings correspond to the Linux distributions), VMWare as Solaris (good luck) and Microsoft as, well Microsoft.

In the case of Windows and Hyper-V, it is actually bundled as one product. We’ll see this happen more and more on the Linux/Xen side as well, as illustrated by Oracle’s offering. I wouldn’t be surprised to see this bundling so common that people start to refer to it as “LinuX” with a capital X.

Side note: I tried to see if the word “LinuX” is already being used but neither Google nor Yahoo nor MSN seems to support case-sensitive searching. From the pre-Google days I remember that Altavista supported it (a lower-case search term meant “any capitalization”, any upper-case letter in the search term meant “this exact capitalization”) but they seem to have dropped it too. Is this too computationally demanding at this scale? Is there no way to do a case-sensitive search on the Web?

With regards to management tools for virtualized environments, I feel pretty safe in predicting that the focus will move from niche products (like those on these lists) that deal specifically with managing virtualization technology to the effort of managing virtual entities in the context of the overall IT management effort. Just like happened with security management and SOA management. And of course that will involve the acquisition of some of the niche players, for which they are already positioning themselves. The only way I could be proven wrong on such a prediction is by forecasting a date, so I’ll leave it safely open ended…

As another side note, since I mention Network World maybe I should disclose that I wrote a couple of articles for them (on topics like model-based management) in the past. But when filtering for bias on this blog it’s probably a lot more relevant to keep in mind that I am currently employed by Oracle than to know what journal/magazine I’ve been published in.

I like XML. Call me crazy but I find it relatively easy to work with. Whether it is hand-editing an XML document in a text editor, manipulating it programmatically (as long as you pick a reasonable API, e.g. XOM in Java), transforming it (e.g. XSLT) or querying an XML back-end through XPath/XQuery. Sure it carries useless features that betray its roots in the publishing world (processing instructions anyone?), sure the whole attribute/element overlap doesn’t have much value for systems modeling, but overall it hits a good compromise between human readability and machine processing and it has a pretty solid extensibility story with namespaces.

In addition, the XML toolbox of specifications is very large and offers standard-based answers to many XML-related tasks. That’s good, but when composing a solution it also means that one needs to keep two things in mind:

• not all these XML specifications are technically sound (even if they carry a W3C stamp of approval), and
• just because XML’s inherent flexibility lets one stretch a round hole, it doesn’t mean it’s a good idea to jam a square peg into it.

The domain of IT management provides examples for both of these risks. These examples constitute some of the technical deficiencies of management-related XML specifications that I mentioned in the previous post. More specifically, let’s look at three instances of XML mis-use that relate to management-related specifications. We will see:

• a terrible XML specification that infects any solution it touches (WS-Addressing, used in WS-Management),
• a mediocre XML specification that has plenty of warts but can be useful for a class of problems, except in this case it isn’t (XSD, used in SML), and
• a very good XML specification except it is used in the wrong place (XPath, used in CMDBf).

Let’s go through them one by one.

WS-Addressing in WS-Management

The main defect of WS-Management (and of WSDM before it) is probably its use of WS-Addressing. SOAP needs WS-Addressing like a migraine patient needs a bullet in the head (actually, four bullets in the head since we got to deal with four successive versions). SOAP didn’t need a new addressing model, it already had URIs. It just needed a message correlation mechanism. But what we got is many useless headers (like wsa:Action) and the awful EPR construct which solves a problem that didn’t exist and creates many very real new ones. One can imagine nifty hacks that would be enabled by a templating mechanism for SOAP (I indulged myself and sketched one to facilicate mash-up style integrations with SOAP) but if that’s what we’re after then there is no reason to limit it to headers.

XSD in SML

The words “Microsoft” and “bully” often appear in the same sentence, but invariably “Microsoft” is the subject not the object of the bullying. Well, to some extent we have a reverse example here, as unlikely as it may seem. Microsoft created an XML-based meta-model called SDM that included capabilities that looked like parts of XSD. When they opened it up to the industry and floated the idea of standardizing it, they heard back pretty loudly that it would have to re-use XSD rather than “re-invent” it. So they did and that ended up as SML. Except it was the wrong choice and in retrospect I think it would have been better to improve on the original SDM to create a management-specific meta-model than swallow XSD (SML does profile out a few of the more obscure features of XSD, like xs:redefine, but that’s marginal). Syntactic validation of documents is very different from validation of IT models. Of course this may all be irrelevant anyway if SML doesn’t get adopted, which at this point still looks like the most likely outcome (due to things like the failure of CML to produce any model element so far, the ever-changing technical strategy for DSI and of course the XSD-induced complexity of SML).

XPath in CMDBf

I have already covered this in my review of CMDBf 1.0. The main problem is that while XML is a fine interchange format for the CMDBf specification, one should not assume that it is the native format of the data stores that get connected. Using XPath as a selector language makes life difficult for those who don’t use XML as their backend format. Especially when it is not just XPath 1.0 but also the much more complex XPath 2.0. To make matters worse, there is no interoperable serialization format for XPath 1.0 nodesets, which will prevent any kind of interoperability on this. That omission can be easily fixed (and I am sure it will be fixed in DMTF) but that won’t address the primary concern. In the context of CMDBf, XPath/XQuery is an excellent implementation choice for some situations, but not something that should be pushed at the level of the protocol. For example, because XPath is based on the XML model, it has clear notions of order of elements. But what if I have an OO or an RDF-based backend? What am I to make of a selector that says that the “foo” element has to come after the “bar” element? There is no notion of order in Java attributes and/or RDF properties.

Revisionism?

My name (in the context of my previous job at HP) appears in all three management specifications listed above (in increasing level of involvement as contributor for WS-Management, co-author for SML and co-editor for CMDBf) so I am not a neutral observer on these questions. My goal here is not to de-associate myself from these specifications or pick and choose the sections I want to be associated with (we can have this discussion over drinks if anyone is interested). Some of these concerns I had at the time the specifications were being written and I was overruled by the majority. Other weren’t as clear to me then as they are now (my view of WS-Addressing has moved over time from “mostly harmless” to “toxic”). I am sure all other authors have a list of things they wished had come out differently. And while this article lists deficiencies of these specifications, I am not throwing the baby with the bathwater. I wrote recently about WS-Management’s potential for providing consistency for resource manageability. I have good hopes for CMDBf, now in the DTMF, not necessarily as a federation technology but as a useful basis for increased interoperability between configuration repositories. SML has the most dubious fate at this time because, unlike the other two, it hasn’t (yet?) transcended its original supporter to become something that many companies clearly see fitting in their plans.

[UPDATED 2008/3/27: For an extreme example of purposely abusing XML technologies (namely XPath in that case) in a scenario in which it is not the right tool for the job (graph queries), check out this XPath brain teasers article.]

On the occasion of the introduction of the Oracle Virtual Machine (OVM) at Oracle World a couple of weeks ago, here are a few thoughts about virtual machines in general. As usual when talking about virtualization (see the OVF review), I come to this mainly from a systems management perspective.

Many of the commonly listed benefits of VMWare-style (I guess I can also now say OVM-style) virtualization make perfect sense. It obviously makes it easier to test on different platforms/configurations and it is a convenient (modulo disk space availability) way to distribute ready-to-use prototypes and demos. And those were, not surprisingly, the places where the technology was first used when it appeared on X86 platforms many years ago (I’ll note that the Orale VM won’t be very useful for the second application because it only runs on bare metal while in the demo scenario you usually want to be able to run it on the host OS that normally runs you laptop). And then there is the server consolidation argument (and associated hardware/power/cooling/space savings) which is where virtualization enters the data center, where it becomes relevant to Oracle, and where its relationship with IT management becomes clear. But the value goes beyond the direct benefits of server consolidation. It also lies in the additional flexibility in the management of the infrastructure and the potential for increased automation of management tasks.

Sentences that contains both the words “challenge” and “opportunity” are usually so corny they make me cringe, but I’ll have to give in this one time: virtualization is both a challenge and an opportunity for IT management. Most of today’s users of virtualization in data centers probably feel that the technology has made IT management harder for them. It introduces many new considerations, at the same time technical (e.g. performance of virtual machines on the same host are not independent), compliance-related (e.g. virtualization can create de-facto super-users) and financial (e.g. application licensing). And many management tools have not yet incorporated these new requirements, or at least not in a way that is fully integrated with the rest of the management infrastructure. But in the longer run the increased uniformity and flexibility provided by a virtualized infrastructure raise the ability to automate and optimize management tasks. We will get from a situation where virtualization is justified by statements such as “the savings from consolidation justify the increased management complexity” to a situation where the justification is “we’re doing this for the increased flexibility (through more automated management that virtualization enables), and server consolidation is icing on the cake”.

As a side note, having so many pieces of the stack (one more now with OVM) at Oracle is very interesting from a technical/architectural point of view. Not that Oracle would want to restrict itself to managing scenarios that utilize its VM, its OS, its App Server, its DB, etc. But having the whole stack in-house provides plenty of opportunity for integration and innovation in the management space. These capabilities also need to be delivered in heterogeneous environments but are a lot easier to develop and mature when you can openly collaborate with engineers in all these domains. Having done this through standards and partnerships in the past, I am pleased to be in a position to have these discussions inside the same company for a change.

When I posted an algorithm for the server side implementation of a CMDBf Query call for version 0.95 of the specification, the interoperability testing session based on that version was over and I was pretty sure no-one but those of us who participated in that session would write an implementation of 0.95. But I published the algorithm anyway since I thought it was helpful to anyone who wanted to understand the specification in depth, even if they were not implementing it. Now that 1.o is out, there is a much higher probability of people implementing the specification, so I figured it would be worth updating the algorithm to take into account the changes from 0.95 to 1.0. So here it is.

One caveat. This algorithm assumes that the query request does not make use of the xpathExpression element because, as I have explained in my review of CMDBf 1.0, I don’t think interoperability is achievable on this feature in the current state of the specification.

As a note of caution, the previous version of the algorithm was backed by my implementation of CMDBf 0.95 for the interoperability testing, so I felt pretty confident about it. For this version of the algorithm I have not written a corresponding implementation and I have not done interoperability testing with anyone, it’s just based on my reading of the specification. The handling of depthLimit in particular is a little tricky and needs to be validated by implementation (what with creating a bunch of dummy item and relationship templates with temporary names and later going back to the original template names), please let me know if you find it flawed.

And, as previously, this is in no way an optimal implementation strategy. It is the most direct and obvious set of steps that I can come up with to implement the Query call in a way that exactly conforms to the specification. There are lots of ways to make this go faster, such as the ones I mentioned in a previous post (e.g. breaking out of loops once an instance has been removed, or not recalculating L1 and L2 over and over again for relationships in the same working set that share a source/target) plus new ones such as being smarter than my brute-force approach to handling depthLimit (in step 2).

All this said, here is the algorithm:

1) for each itemTemplate, calculate the set of all items (including relationships since they are a subclass of item) that obey the instanceIdConstraint and recordConstraint elements in the template (if present). Call this the working set for the itemTemplate.
2) for each relationshipTemplate RT that has a depthLimit element:

2.1) for i ranging from 1 to the value of maxIntermediateItems for RT:

2.1.1) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s sourceTemplate, except that it has a new, unique temporary id (keep a record linking that new id to the id of the original source itemTemplate).
2.1.2) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s targetTemplate, except that it has a new, unique, temporary id (keep a record linking that new id to the id of the original target itemTemplate).
2.1.3) for j ranging 1 from i:

2.1.3.1) create an itemTemplate that is an exact copy of the itemTemplate referenced by RT’s intermediateItemTemplate, except that it has a new, unique, temporary id (keep a record linking that new id to the id of the original intermediary itemTemplate).
2.1.3.2) create a relationshipTemplate that is an exact copy of RT, except that its source is the itemTemplate created in the previous iteration of the current loop (or the itemTemplate created in step 2.1.1 if j=1), its target is the itemTemplate created in the previous step and it has a new, unique, temporary id (keep a record linking that new id to RT’s id).

2.1.4) create a relationshipTemplate that is an exact copy of RT, except that its source is the last itemTemplate created in the 2.1.3 loop, its target is the itemTemplate created in 2.1.2 and it has a new, unique, temporary id (keep a record linking that new id to RT’s id).

3) for each relationshipTemplate calculate the set of all relationships that obey the instanceIdConstraint and recordConstraint elements in the template (if present). Call this the working set for the relationshipTemplate.
4) set need_to_loop = true
5) while (need_to_loop == true)

5.1) set need_to_loop = false
5.2) for each relationshipTemplate RT

5.2.1) let ITsource be the itemTemplate that is referenced as sourceTemplate by RT. Calculate the set of all items (including relationships since they are a subclass of item) that obey at least one of the instanceIdConstraint elements in ITsource (assuming there is at least one such element) and all the recordConstraint elements in ITsource. Call this the working set for ITsource.
5.2.2) let ITtarget be the itemTemplate that is referenced as targetTemplate by RT. Calculate the set of all items (including relationships since they are a subclass of item) that obey at least one of the instanceIdConstraint elements in ITtarget (assuming there is at least one such element) and all the recordConstraint elements in ITtarget. Call this the working set for ITtarget.
5.2.3) for each relationship R in the working set for RT

5.2.3.1) if the source of R is not in the working set for ITsource, then remove R from the RT working set
5.2.3.2) if the target of R is not in the working set for ITtarget, then remove R from the RT working set
5.2.3.3) if RT has a source/@minimum or a source/@maximum attribute

5.2.3.3.1) find the list L1 of all relationships in the working set for RT that have the same source as R
5.2.3.3.2) if RT has source/@minimum and the cardinality of L1 is less than this minimum then remove all relationships in L1 from the RT working set
5.2.3.3.3) if RT has source/@maximum and the cardinality of L1 is more than this maximum then remove all relationships in L1 from the RT working set

5.2.3.4) if RT has a target/@minimum or a target/@maximum attribute

5.2.3.4.1) find the list L2 of all relationships in the working set for RT that have the same target as R
5.2.3.4.2) if RT has target/@minimum and the cardinality of L2 is less than this minimum then remove all relationships in L2 from the RT working set
5.2.3.4.3) if RT has target/@maximum and the cardinality of L2 is more than this maximum then remove all relationships in L2 from the RT working set

5.3) for each itemTemplate IT:

5.3.1) let sourceRTset be the set of all relationshipTemplates that references IT as its sourceTemplate
5.3.2) let targetRTset be the set of all relationshipTemplates that references IT as its targetTemplate
5.3.3) for each item I in the IT working set

5.3.3.1) for each relationshipTemplate sourceRT in sourceRTset, if there is no relationship in the working set for sourceRT that uses I as its source, remove I from the IT working set and set need_to_loop to true
5.3.3.2) for each relationshipTemplate targetRT in targetRTset, if there is no relationship in the working set for targetRT that uses I as its source, remove I from the IT working set and set need_to_loop to true

6) process the eventual contentSelector elements and/or the @suppressFromResult attributes on the templates that have matching items/relationships in the response to remove or pair down items and relationships as requested
7) package the resulting items and relationships in a way that conforms to the CMDBf response message format (including putting each item in the <nodes> element with the appropriate @templateId attribute and putting each relationship in the <edges> element with the appropriate @templateId).
8) replace all the temporary template ids (from step 2) that appear in templateId attributes in the response with the original ids of the items and template based on the records that were kept in step 2.

Just to clarify things, what I do in step 2 is simply make explicit all the itemTemplates and relationshipTemplates that are made implicit by the depthLimit element, so that we can provide with a simpler algorithm after that assumes that all relationshipTemplate correspond to direct relationships (no intermediary). And in step 8 I hide the fact that this took place.

Microsoft introduced an umbrella project called Oslo at their SOA and Business Process conference this week. There is very little information available but it seems to have two main components: improving the ability of the Microsoft platform to support SOA-style distributed applications and improving the use of models to develop and manage applications. At first sight there isn’t anything new. The SOA talk is similar to any number of “why SOA” presentations available from dozens of companies. And the modeling aspect is the same story that Microsoft has been pitching with DSI for years. The real news is that the two stories are being linked (at least at the marketing level, which is a starting point) and that the application development people have taken over the application modeling baton from the System Center group.

Over the last few years, I worked with people from System Center on different standards related to DSI, including SML which they see as the heart of the modeling effort. One of the things that kept me skeptical when hearing the DSI pitch, was to see the System Center team making announcement and promises about how SML would be central to the development experience in Visual Studio. I am pretty sure I know who’s the gorilla and who’s the chimp at Microsoft between Visual Studio / .Net Framework on the one hand and System Center on the other. The application model is too central to the developer experience for the Visual Studio group to not own it. It looks like it’s now happening and it’s a good thing.

The only content I could find on Oslo that’s not PR fluff is a report from Directions on Microsoft which mostly talks about incremental improvements to BizTalk. Towards the end, there is a small section about a “repository” that will “provide centralized storage of composite application components”. At that point I can’t help remembering the blog post from David Chappell about why it wouldn’t make sense for Microsoft to support SCA. Through comments in his post as well as a blog post of my own, I followed-up with the assertion that the application component model also plays a very important role for management. And at the risk of sounding self-congratulatory, the Oslo announcement seems vindicate that view. I see that David was a speaker at the Microsoft conference where Oslo was announced and he has very good insights into both the application developement and the systems management efforts at Microsoft. So hopefully he’ll soon have a white paper or a blog entry out to share some insights.

If you’re wondering what this means for the technical work that has been going on under the DSI umbrella so far, you can only read the tea leaves. It could be that the application development people adopted the whole SML/CML technology stack as promoted by their System Center colleagues and are going to use it as is. Or on the other extreme, it could be a complete reset that leads to the creation of a component model that is much less general and much more application-centric. Of course, no matter which one happens (or something in the middle), it will be presented as a perfectly smooth and controlled evolution of the DSI vision (get ready for some nice spin at MMS2008). If you are adopting SML because you expect Microsoft to base its application component model on it, you might want to wait a bit until more details emerge about Oslo. For example, after calling XSD “a schema language that attempts to be a floor wax, dessert topping, and personal lubricant all at the same time” you have to wonder whether Don Box would advocate to use SML (80% of which is XSD) as the most effective metamodel for an application component model…

Let’s end with this quote from the Directions on Microsoft report on Oslo, regarding application integration: “SAP and Oracle are better positioned in that regard, and so their customers will want to investigate these vendors’ composite application platforms along side Microsoft’s”. Can’t disagree with that. A good place to start this investigation would be the upcoming Oracle Open World.

Having read the recently released CMDBf 1.0 specification over the weekend, I see several improvements since 0.95, including:

• the introduction of depthLimit
• the lastModified metadata element
• the ability to specify more than one instanceId in a template
• the ability to advertise what parts of the specification you implement
• the definition of faults

But while 1.0 is more complete than 0.95, I think it makes it harder to achieve interoperability. Here are the main friction points for interop:

New role for XPath

The xpathExpression element (which replaces xpath1Selector) changes in two very important ways. First, rather than being limited to XPath 1.0, it now also allows XPath 2.0. Support for this is a lot harder to achieve for people who don’t use XML as the backend format for their data. Considering the current state of adoption of XPath 2.0 and the low level of XML complexity exposed by most CMDB models, I don’t think it was opportune to bring this into CMDBf yet. And my guess is that most implementations will stay away from this. But there is a second change, less obvious but even more problematic. XPath is not just another constraint mechanism for a CMDBf template anymore, one that returns a boolean result indicating whether the instance meets the constraint or not, as it used to be in 0.95. It is now an alternative selection and filtering mechanism that lives in parallel to all the other elements in a template (and can’t mix with them). Overall, I think this change goes too far in the direction of turning a shared agreement to exchange data in XML into an assumption that the internal data models are all based on XML. And the killer with regards to interoperability is that the specification says nothing about how the resulting node sets are serialized in the response. There may be a serialization for the XPath 2.0 model, but there is no such thing for XPath 1.0 and I don’t see in the current state of the specification how two implementations have any chance to interoperate when using this feature.

Introduction of linkDepth

As I mentioned earlier, linkDepth is a very useful addition (even though it pales in comparison to the inferencing capabilities that could have been derived from basing CMDBf on RDF). But it is also a complicated feature. The intermediateItemTemplate attribute is a good re-use of the existing plumbing, but it needs at least a detailed example. I trust that the group will generate one once they’ve caught their breath from putting out the specification.

Service capability metadata

There is a new section (#6) to provide ways to describe what CMDBf features an implementation supports. But it is a very granular representation. Basically, for every feature you can describe if you support it or not. So someone may describe that they support everything inside propertyValue, except for the “like” operator. And someone else might support all the operators but not the caseSensitive modifier. That might be ok for human consumptions, but automated scenarios rely on pre-programmed queries and that is made very hard by all the possible combinations of options. What we need is a few well-defined profiles that people implement fully. Starting of course with a profile that rules out xpathExpression.

Record metadata

This new version introduces metadata on records. While recordId and lastModified are probably well understood and interoperably usable I am a bit more dubious about whether baselineId and snapshotId are going to be interoperable across vendors based on their limited description in the specification. The nice thing is that this metadata can not only be returned but also searched on. Well, at least that’s the intent. But this goes through the recordMetadata attribute on propertyValue which, while present in the pseudo-schema, is missing in the XSD…

The contentSelector element

This new element is more flexible that the propertySubsetDirective element that it replaces. In addition to specifying what properties you want returned it also allows you to specify that you only want certain record types and/or that you only want the record(s) that were used to satisfy constraints in the template. Those are nice additions, but the way the second part is implemented (through the use of the matchedRecords attribute) seems to assume that only one record in the instance was used to match all the constraints in the template. This is not necessarily the case, an instance can be selected by having different records match the different constraints in the template as long as it has at least one matching record per constraint (line 765 says “the item satisfies all the constraints”, not “a record of the item satisfies all the constraints” and you can also see this in the example in section 4.2 where the records mentioned on lines 637 and 639 don’t have to be the same). So do you return all records that have a role in matching the template, or only those (if there is any) that matches all the constraints on their own as the text seems to imply? And if several record combinations inside an instance can be used to match the constraints in a template, do I return all of them or can I just pick any subset that matches? Also, how can I say that I want all records that established the template match, independently of their type? There doesn’t seem to be a way to do this, or is it by putting a contentSelector element with no child element and the matchedRecords attribute set to false? There won’t be much interoperability on this feature until all this is clarified.

Relationships as items

A major change between 0.95 and 1.0 is that now a relationship can match an itemTemplate. For example, if you ask for all items that were modified during the last 24 hours you will get all the items and all the relationships that meet that criteria while in the previous version you’d have to explicitly request the relationships with a relationshipTemplate if you wanted to get them too). There is a good case to be made for either view and the one that works best largely depends on your backend implementation technology (RDF, objects, SQL, CIM…). But the important thing is for the spec to be clear and on this point I think the change wasn’t made explicit enough in the query section of the specification. If Van hadn’t called my attention to this on his blog, I would have missed this important change.

Security boilerplate

There is a person at IBM (probably located in a well-stoked underground bunker in upstate NY) who has instilled the fear of god in all IBM employees (at least all those who author publicly available specifications) and forces them to include a boilerplate “security considerations” section everywhere. I have co-authored several documents with IBM employees and it never fails, even thought it doesn’t add anything useful to the specification. You should see the look of fear on the face of the IBM employees when someone else suggests doing without it. We somehow managed to sneak one such slimmer specification past the IBMers with CMDBf 0.95 but I see that this has been “corrected” in 1.0. I hope that whatever painful punishment Scott, Jacob, Andrew and Mark (or their families and pets) were subjected to in the process by the IBM security ogre wasn’t too cruel. Sure, this doesn’t really impact interoperability, but now that I don’t work for a company that makes money from ink anymore, I have even less patience for this bloating.

OK, that’s enough back seat driving for now. Hopefully the standards group that will take over the specification will address all these questions. In the context of the entire specification, these are pretty small issues and mostly easy to fix. And the CMDBf group can go on to address the hard issues of federation (including security-related issues that abound in this field if one really wants to tackle them). The current specification is a useful graph-oriented query language that is a good match for CMDB data. But it’s really just a query language (plus a simple registration system).

[UPDATE: while updating the CMDBf query algorithm, I noticed another small error: maxIntermediateItems is an attribute in the pseudo-schema but an element in the schema. Something else to fix in the next version.]

The CMDBf committee has just released version 1.0 of the specification. Van Wiles has an overview of the changes between 0.95 and 1.0. I left HP soon after 0.95 was released and that’s when my participation in CMDBf ended, so Van’s summary is very useful to me. The changes he lists are not surprising and some of them already existed in draft form before 0.95 publication. I need to spend some intimate time with the specification to review the changes to the template mechanism in more details. Some of the changes have the potential to make the specification quite a bit harder to implement. This is especially the case for the introduction of “depthLimit” (but it’s probably a needed feature anyway). And the fact that relationships can now match item selectors will make things either easier or harder to implement, depending on your implementation choice (e.g. straight-to-SQL/XML or through an OO or RDF model). Congrats to the group. We should soon hear about submission for standardization.

Dave Chappell and David Berry have recently published an article in SOA Magazine titled “The Next-Generation, Grid-Enabled Service-Oriented Architecture”. I had unexpectedly gotten a quick overview of this work a few weeks ago, when I ran into Dave Chappell at the Oracle gym (since I was coming out of an early morning swim it took Dave a couple of seconds to recognize me, as I walked through the weight room leaving a trail of water behind me). Even if you are more interested in systems management than middleware, this article is worth your reading time because it describes a class of problems (or rather, opportunities) that cut across middleware and IT management. Namely, providing the best environment for scalable, reliable and flexible SOA applications. In other words, making the theoretical promises of SOA practically achievable in real scenarios. The article mentions “self-healing management and SLA enforcement” and it implies lots of capabilities to automatically provision, configure and manage the underlying elements of the Grid as well the SOA applications that make use of it. Those are the capabilities that I am now working on as part of the Oracle Enterprise Manager team. And the beauty of doing this at Oracle, is that we can work on this hand in hand with people like Dave to make sure that we don’t create artificial barriers between middleware and systems management.