William Vambenepe's blog

Apparently Coté’s upstairs neighbors were having a party last night and he could not sleep. That’s good for us because as a result he bookmarked a long list IT systems management stories. Several of those picked my interest:

• Is User Monitoring The Next Wave In Enterprise App Management?
  Nice overview. Forgets to mention Moniforce (now Oracle UXInsight). Also, when they say that “it doesn’t get any easier than installing a bunch of appliances in your network”, I’d like to see them install an appliance that decrypts HTTPS traffic in a bank. Tell me how easy that was.
• Network and IT management platforms : 2007: Products of the Year: SearchNetworking.com
  How about this as a PR challenge for HP. A silver award is not shabby but how do you spin the fact that the gold went to a free product?
• Mapping HP Software to the Data Center Automation Blueprint
  Nice overview. Quite a few changes (at least at this high level) since I left, mostly from Opsware integration. WRT the security question at the end, don’t hold your breath. But you never know. How many Ping could you buy for the price of one EDS?
• Microsoft targets CA, HP with new management attack - Network World
  Already commented on this here. I doubt anybody at HP is surprised.
• Jeff’s Professional Side » Blog Archive » Vendors, Open Source, and Hypocrisy
  As I already commented at the bottom of this post (look for the “UPDATED 2008/5/15″ section), “if a given SNMP agent works with HP, IBM, BMC and CA you will probably save yourself time in the long run by finding a way to support it (even if it is not spec-compliant) rather than getting the vendor to change”.

With the IBM/Microsoft/Intel/HP WSDM/WS-Management convergence now implicitly (if not yet officially) dead, it will be interesting to see what IBM is going to do with WSRF. WSRF is being used today, rarely explicitly but rather in an embedded fashion. People who use WSDM use it, people who use CDDLM use it, people who use the Globus Toolkit use it, etc. IBM could write off the convergence work (WS-ResourceTransfer, which was published as a draft, and WS-ResourceEnumeration and WS-EventNotification which were never published) and stick to using the existing WSRF specifications when they need the corresponding functionality. That’s what I hope they do.

Alternatively, they could decide to get the forceps out of the drawer. They can create a new, IBM-friendly (e.g. Fujitsu, CA, Cisco…) private consortium to take over the unfinished drafts (if the IBM/Microsoft/Intel/HP legal agreement allows this) or start new ones. Or they could go directly to W3C, OASIS or OGF and push for a new working group to do the work in the open (and since no-one else would really care about this work IBM should have relatively free hands there, the way Microsoft did in DMTF when IBM chose to boycott WS-Management). Why W3C would care and why OASIS or OGF would want to start commitees to obsolete their existing work is a separate question.

While I hope that IBM doesn’t try to push another pile of WS-* resouce management specifications on an industry that already has too many, if they do I hope that at least they’ll do it right. And that means doing away with the approach embedded in WS-ResourceTransfer. Having personally been involved in many iterations on this problem, I hope to have some insight to contribute.

Along the lines of the age-old parental advice “don’t do it but if you are going to do it then use a condom”, here is my advice to anyone thinking of doing another iteration on the WSRF question: don’t do it but if you are going to do it then be specific about what problem you are addressing.

First, let’s separate three scenarios.

Database query

WS-ResourceTransfer should not be seen as a way to query an XML database. Use XQuery for this.

REST

While architecturally it should be possible to build RESTful applications on top of WS-Transfer’s operations, this is simply not what is happening. WS-Transfer is being used either by CIM people (who get to it via WS-Management) or by big-SOA people (who get is as part of the whole WS-* stack) and neither of them is doing anything remotely RESTful. So just leave that aside and don’t see WS-ResourceTransfer as a way to do “fine-grained REST”. No REST user is loosing sleep over WS-ResourceTransfer being in limbo.

A flexible way to interact with a complex system

This is the use case that you should focus on. You have a system made up of many parts (e.g. a composite application or a server that is made of many components) that you can represent as an XML document. The XML repesentation contains some important information about the system, but it isn’t the system. There are identified resources within the system that have lifecycles, management capabilities and internal parameters. Not everything relevant is captured in the XML model. This is why it is different from an XML database.

In general, I don’t think that XML is the best way to represent complex IT systems. It has plenty of complications that are not relevant to IT management and it doesn’t elegantly support the representation of graphs, often the most natural way to represent such a system (more on this here). CMDBf, with its graph-oriented approach, is a better choice in general. But there are plenty of areas (especially smaller, well-defined, sub-systems) in which XML formats have been defined to represent systems. SCA and SML for example.

In the case where you are dealing with such an XML-described system, then there is value in standard ways to simplify interactions with the system and its parts. But here too, we need to distinguished different patterns rather than trying to handle them all in the same way.

Filtering/sequencing of returned data

Complex IT systems can generate a lot of configuration and/or monitoring data and often you only care for a small subset. For example, an asset record has dozens of elements (lease terms, owner, assigned user…) but you may only care to retrieve the date the lease expires. When you do a GET on the record, you want to qualify it by specifying that only that date needs to be returned. That’s what WS-RP, WS-RT and the WS-Management wsman:TransferFragment header allow. In a variation of this, you want all the data but you don’t want it in one go, you want to pull it piece by piece. That’s what WS-Enumeration gives you. The problem with all these specifications is that they only offer that feature when you are retrieving the resource representation (a WS-Transfer GET or equivalent), not for other operations. But how is this different from invoking an AirlineBooking operation and saying that you only want to be sent the confirmation code, not the full itinerary, equipment type, assigned seat, etc? Bundling this inside WS-RT (or equivalent) is not helpful. A generic SOAP header that can go on any message would be more appropriate (the definition of this header would need to pay special attention to security considerations, especially if the response is signed, because it could be abused to trick the server into sending, and signing, specifically-crafted messages).

Interacting with a sub-element of the system

If you have a handle to a computer system resource and you know that it has one CPU and that this CPU is represented by the /comp:CPU element of the system, why would you need to use some out-of-band discovery mechanism to interact with that CPU? It’s right there, you can see it, you can point to it. Surely there must be a way to address operations to it directly, right? WS-Management tries to do it with its wsman:Selector mechanism, but the selectors are not tied to the model and require, effectively, a separate out-of-band agreement for addressing. There shouldn’t be a need for such an additional agreement once an agreement has already been reached on the model.

What is needed is a way, for systems that have a known XML model, to address message to subpart by using the model itself to support that addressing. Call it SOAPy mashup if you want to feel like you are part of the cool kids. I described such a mechanism a while ago. In effect, it is an improvement on wsman:Selector that an eventual new iteration of WSRF should at least consider.

In some cases, namely when the operation is a WS-Transfer GET, this capability overlaps with the “filtering of returned data” capability. One way to look at it is that you are doing a GET at the level of the overall computer system and filtering the results down to the part that represents the CPU. Another way to look at it is that you are pinpointing the message to a subset of the model (the CPU part) and doing an unmodified GET on it. It doesn’t matter how you choose to think about it. In my proposal, these two ways produce the same message. Like the wave view and particle view of a photon, that in the end, describe the same physical entity with each being the best representation for a set of situations.

The problem with WS-RT and its predecessors is that it doesn’t recognise that this is just the intersection of two orthogonal concerns (filering of output versus addressing of sub-elements) and only handles that intersection.

Interacting with a set of resources as a set

The same kind of expression (typically XPath) that lets you point at a sub-element inside of a system also lets you point at a set of such sub-elements. But even though from an XPath perspective there isn’t much of a different (the first one just happens to return a nodeset that contains only one node), from an architectural perspective it is a very different use case. If you want to support such a use case then you have handle it as such and define all the associated semantics (sequential/parallel execution, fault handling, partial completion, resource-specific permissions…). You can’t just cross your fingers and assume that you get such features “for free” just because XPath can return a nodeset.

I know that this post illustrates a way of giving free advice that virtually ensures that it gets ignored. Similar (if you’ll allow the big stretch) to the way Chirac and Villepin were arguing againt an Iraq invasion in ways that probably reinforced the Bush administration’s determination to do it. When will the world finally learn to appreciate the oh-so-slightly obnoxious undertone that is inherently French (because, let me tell you, we’re not about to loose it)? At least, when my grandchildren ask me “where were you when IBM invented WS-ManagementHammer?” I can point to this post and say “I tried to stop it, I tried”.

[UPDATED 2008/5/15: How timely! Just after publishing this I find, via Coté, what looks like another example of French abrasiveness in the systems management world: the attitude, name and the way Jeff ends with a French-language quote make it quite likely that the "Jacques" person discounting the fact that his company's SNMP agent is broken is indeed a compatriot. French obnoxiousness aside, and despite my respect for standards, my advice to Jeff is that if a given SNMP agent works with HP, IBM, BMC and CA you will probably save yourself time in the long run by finding a way to support it (even if it is not spec-compliant) rather than getting the vendor to change. There are lots of sites out there that work fine with Firefox and IE but are not compliant with Web standards. Good luck getting them all fixed.]

I missed this good review of Oracle Enterprise Manager (OEM) by eWeek’s Cameron Sturdevant that came out almost two months ago. It is “good” in the sense that it is well researched and well written but it is also “good” in the sense that it is a very positive review. The only drawback listed is the price of some of the features. But you have to evaluate these numbers in comparison to productivity gains of your IT management staff. Or, even more compellingly, in comparison to the cost of business disruption that can result from insufficient management insight into the applications.

I got to this review through this very nice blog post in which my colleague Chung Wu (a director of product management for OEM) describes step by step the key role that OEM plays in effectively managing Oracle technologies and in allowing a smooth and controlled evolution of the deployed portfolio.

A couple of product updates related to applications management were announced over the last couple of weeks:

• My ex-colleagues at HP working on SOA management have released a new version of SOA Manager (the product that originated with the TalkingBlocks acquisition, when coolness first entered the gloomy 42-Lower floor of HP Cupertino) plus some SOA-buzzword-compliant improvements to Mercury-inherited products (testing tools and BAC). Or so at least says this article (I couldn’t easily find any specifics on the HP site).
• The JBoss guys announced last week version 2.0 of JBoss ON (Operations Network) their application management console. I assume it is a follow-on to the previously announced work with Hyperic even though the press release does not mention anything about it.

I was hoping that by the time MMS was over there would be more clarity about the “Cross Platform Extension” to System Center that Microsoft announced there. But most of the comments I have seen have focused on two non-technical aspects: Microsoft is interested in heterogeneous management and Microsoft makes use of open source. That’s also the focus of Coté’s coverage.

So what? Is it still that exciting, in 2008, to learn that Microsoft recognizes that Linux and OSS are major players in enterprise computing? If Steve Ballmer eventually gets hold of Yahoo, do you think his first priority will be to move all the servers to Windows or to build up its search and advertising audience? It’s been now 10 years since the Halloween documents came out. They can be seen as the start of Microsoft’s realization that Linux/OSS are here for good. It is not surprising to see that one of their main authors is now the driving force behind WS-Management, an effort that illustrates the acceptance of heterogeneity and the need to deal with it (on Microsoft’s terms if possible, of course). The WS-Management effort started years ago and it was a clear sign that Microsoft knew it had to tackle heterogeneous management (despite the reassuring talk that “it’s all about making Windows the most manageable platform” to HP and others). Basically, Microsoft is using WS-Management to support heterogeneity without having to do too much work: by creating an industry standard that everyone writes to and that Microsoft uses internally. Heterogeneous management is intrinsic to DSI if DSI is to be anything more than a demo.

But all of this was known before MMS 2008 to anyone who was paying attention. Instead of all this Microsoft/OSS/heterogeneous talk, I am a lot more interested in the technical aspects of the “Cross Platform Extension”.

OpenPegasus has been around for a long time, as a C++ CIMOM with a bunch of associated providers and CIM-XML interoperability over HTTP with CIM clients. I don’t know where WS-Management support was on the OpenPegasus development timeline, but even without Microsoft getting involved it would have eventually happened. And this should have been sufficient for System Center to access the CIMOM (BTW, does System Center not support CIM-XML when WS-Management is not present and if it does then what is different in practice with WS-Management?).

I can see how Microsoft would bring some extra (and much welcome) development resources for the WS-Management implementation (BTW the guys at Intel already have an open-source C implementation of WS-Management) as well as some extra marketing/visibility/distribution. Nice, but not earth-shattering. Do they bring anything else to OpenPegasus?

And what else is in the “Cross Platform Extension” in addition to an OpenPegasus WS-Management-capable CIMOM? Is there any extra modeling capability beyond CIM? Any Microsoft-specific classes? Any discovery/reconciliation capability? How much actual configuration management versus just monitoring? Security? Health models? Desired state management? Or is it just a WS-Management CIMOM? Any pointer to specific information is welcome.

Of course the underlying question is whether others than Microsoft can manage resources that have an OpenPegasus-based System Center management pack on them. The Open Management Consortium guys have talked about an open management agent. Could, against all expectations, Microsoft be the one delivering it?

In the IT management world, there are the big 4 (HP, BMC, CA and IBM), the little 4 (Zenoss, Hyperic, GroundWorks and openQRM) and the mighty 3 (Oracle, Microsoft and EMC). Sorry John, I am reclaiming the use of the “mighty” term: your “mighty 2″ (or 2.5) are really still the “little 2″ (or 2.5). At least for now.

The interesting thing is that in that industry configuration there are topics on which the little ones and the mighty ones share common interests. For example, the big 4 have a lot more management packs for all kinds of resources, built up over the years. Some standard-based mechanism that partially resets the stage helps the little ones and the mighty ones better compete against the big 4. Even better if it has an attractive (and extensible) implementation ready in the form of an agent. But let’s be clear that it takes more than a CIMOM to make a management pack. You need domains-specific expertise in the form of health models, deployment/configuration scripts and/or descriptors, configuration validation, role management etc. Thus my questions about what else (beyond CIM over WS-Management) Microsoft is bringing to the table. SML and CML are supposed to address this space, but I didn’t hear them mentioned once in the MMS coverage.

[UPDATED on 2008/5/7: Another perspective on Microsoft and open source: Microsoft Ex-Pats Developing Open Source Software Outside of Redmond]

[UPDATED 2008/5/7: I got an answer to the question about System Center support for CIM-XML: it doesn't have it. So indeed it's either WS-Management of WMI. If you're a Linux box, that means it's WS-Management.]

A few announcements came out today.

The good news: Oracle’s acquisition of BEA closes. Unobstructed technical work can start.

The conveniently-timed news: WS-Management officially a standard.

Speaking of MMS 2008, any announcement there? Not much so far, as explained by Ian Blyth. If I parse the cross-platform part of the press release correctly, it says that management of non-Windows resources by Operations Manager is based on WS-Management, but WS-Management alone is not enough so Microsoft is providing a development kit for several non-Microsoft operating systems. It will be interesting to see what exactly is produced by these management packs. Can they be called on by management tools other Operations Manager or is the stuff that rides on top of WS-Management too proprietary to allow this? No word on SML/CML.

By the end of the week we may have a clearer picture, including what’s going on with the previously-announced reset on System Center Service Manager. Coté is on the scene and will undoubtedly share his thoughts.

As a side note, the way the MMS main page loads betrays the fact that, in 2008, Microsoft (or more likely its event marketing contractor) is using the same clueless HTML design approach that I first saw in 1995 and recently wrote about. All the text in the center of the MMS home page is contained in one large picture (available here). They didn’t even bother with a “ALT” field, so good luck to blind users. The part that says “Registration Overview Page” was made blue and underlined to suggest that it is a link, but it is just a part of the picture. Which, presumably, was supposed to be turned into a link using an image map. Well, turns out they can’t even get that right.

They tried to use a client-side image map (not available in 1995) but somehow the actual map code is commented out in the HTML source:

<!–<map name=Map>
  <area shape=RECT coords=18,549,210,572 href=”registrationoverview.aspx”>
  <area shape=RECT coords=17,596,222,634 href=”registrationoverview.aspx”>
</map>–>

As a result, the single most preeminent link on the home page is dead. And there is no server-side image map mechanism as a backup (which I remember used to be best practice when client support for client-side image maps was spotty).

Looking at the HTML source also reveals that tables are over-used. That’s the kind of HTML I can write, and I don’t mean that as a compliment.

[UPDATED 2008/5/5: As expected/hoped, Coté did share his thoughts on this "cross-platform" move from the MMS floor.]

Spring’s Rod Johnson writes today about the future he sees for Java Bloatware (his unkind term for Java EE middleware). Of course, as Mr. Spring, he is far from neutral. Of course he is focusing on a certain class of applications (web-centered, mostly greenfield, which is a huge - and sexy - segment, but not in any way the only kind of applications). Of course he underestimates how established technology that works remains used long after it may have ceased to be the optimal solution for new developments. But even taking all that into account, he makes some good points about the proliferation of rarely-used capabilities in Java EE and the associated cost. Most of those points are well understood and are driving the more modular approach taken by Java EE 6. As well as the adoption of OSGi (see here and here for BEA’s example). In addition, as Rod mentions, the JCP now has to share the playground with other framework standardization efforts like SCA.

The most interesting part of Rod’s post from my perspective, is this prediction:

“The market will need to address the gap between Tomcat and WebLogic/WebSphere. Currently an important part of the market is neglected. The majority of Java web applications are most at home on Tomcat. A minority actually want some of the more esoteric functionality of a full-blown application server, such as JCA, or specialized capabilities such as distributed transaction management. But a larger minority need some of the operational and management features of those products, but are not interested in the esoteric APIs and the bloat they bring along with them. As more and more end user companies look to phase out legacy application servers in favor of better suited technologies, there will inevitably be a response to market demand, with products that hit the sweet spot and bridge this gap.”

Right on. This is the second time in a week that we see an acknowledgment of the importance of application manageability coming from SpringSource. Whether this mid-point demand will be met from the top down by a more modular Java EE stack or from the bottom up by building on top of Tomcat (or some non-Java HTTP server) remains to be seen. The two aren’t exclusive either.

I expect that the hosted application frameworks like the recently announced Google App Engine will also aim at that “more than Tomcat, less than J2EE” sweetspot. But the cost/benefit formula of a more full-featured (or “bloated” if you prefer Rod’s terminology) environment might turn out to be different in a “hosted framework” situation.

SpringSource has made some recent announcements, in an effort to build up its commercial offering on top of the open source Spring framework. There is now a SpringSource Enterprise subscription which gives you access to an “enterprise” edition of the framework, some support and the SpringSource Performance Suite.

The first two components (enterprise edition and support) are common approaches to commercial open source.

The performance suite is a new product, comprised of the Tool Suite (for development), an Advanced Pack for Oracle (for better use of Oracle RAC features) and the Application Management Suite (AMS). Application and middleware management is what I care most about, so AMS is the part of the announcement that caught my attention.

The only publicly-accessible source of meaningful information about AMS that I could find is this blog post by Jennifer Hickey. AMS is built on Hyperic. The monitoring is based on collecting, through instrumentation, entry and exit times for monitored methods. The agents then reports this to a server. Add to this some discovery capabilities and the console can then report observed metrics on the discovered/selected resources.

The blog post ends by saying that “we’d like to make it as powerful and easy to use as possible for both Developers and Operations staff”. At this stage, I think it’s a lot more likely to be used for development than for operations. The instrumentation overhead is supposed to be “very slight” but, as always with monitoring, this warrants more precise data. Also, it is not clear if/how AMS can integrate with other management tools.

In any case, it’s encouraging to see an open source application development framework which doesn’t entirely focus on ease of development but also acknowledges the full lifecycle of an application (and concerns such as monitoring, as addressed here, but also configuration management, governance, business activity management…). That’s the difference between “the best framework to create an application” and “the best framework to create an application that is expected to be used”. Before open source became a business strategy, a defining characteristic was that the developers where also users of the product. Which naturally meant that it was heavily biased towards developers and development tasks.

From an operations perspective, the AMS team should focus its efforts on application modeling, metric collection and management integration rather than the dashboard. A simple specialized console is great for application developers. The ability to discover, model, configure and monitor applications in conjunction with the other elements of the IT system (e.g. underlying infrastructure, end user experience, business processes and other forms of application integration, etc) is what operators really need.

In any case, it will be interesting to test the practical value of “Spring-aware” application management, above and beyond generic Java application management.

Bonus question: the enterprise edition of the Spring framework is “warranted to be virus-free”. Since the enterprise version includes the base framework, to the extent that the enterprise version is virus-free then mustn’t the base logically be “virus-free” as well? And what does “virus-free” mean anyway?

Somewhat lost in the news about Oracle’s recent earning report is the announcement that Oracle just purchased e-TEST suite from Empirix. We are not purchasing the company, just some of their products (they also sell VoIP testing tools, for example, which will stay with Empirix). Most importantly, we are also getting the people who made the product, not just a code dump. They’ll join the Enterprise Manager team (my group). Welcome aboard!

The e-TEST suite is made of three integrated components (I am describing the current e-TEST suite, not necessarily the resulting Oracle offering):

• e-Manager Enterprise is a process management application for application testing.
• e-Tester lets you easily create sophisticated tests for functional and regression testing.
• e-Load is a load and performance testing framework.

(these product names make me feel like I am back in HP’s e-speak team)

This is a mature product suite that will increases the scope and depth of EM’s application testing capabilities. It extends the existing EM recorder/beacon infrastructure. It offers a sophisticated test transaction model (remember VBA?). It offers load testing capabilities. Not to mention the process management capabilities around test cases.

My toddler daughter loves her book about the solar system. She has learned to say “hot!” whenever we look at Mercury. Tonight I’ll have to teach her to say “feeling the heat” instead.

More info here. That is probably also where specific product plans will be released.

If you’ve ever been to an Oracle Open World presentation, you won’t be surprised to see that this post ends with a disclaimer that:

It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality and should not be relied upon in making a purchasing decision. The development, release and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

In his 15 Ways to Tell Its Not Cloud Computing post, James Governor asserts that:

“If you know where the machines are… its not a cloud.”

I took issue with this in a comment on his post.

And today, Amazon EC2 makes me feel smug:

“Availability Zones give you additional control of where your EC2 instances are run. We use a two level model which consists of geographic regions broken down into logical zones.”

Here are more details on how it works. And Amazon’s feature guide for availability zones.