William Vambenepe's blog

Many of us have been thinking (a bit) and talking (a lot) about the relationship between Clouds and good old IT management.  John understands both sides and produced a few good posts (like this one).

Maybe it’s just a coincidence that both Hyperic and CA recently made such announcements. In any case, it gives the impression that time has come for some actual product capabilities in the area of managing Cloud-based systems.

I haven’t investigated either, so keep your slideware shields up, but this is what I read:

From Javier Soltero’s “Announcing HQ 4.0″: “It also provides the first cloud-friendly management agent which allows users to manage cloud based virtual machines securely and reliably from either inside the cloud, or from HQ 4.0 installations inside your datacenter”. John approves.

And at CA World, according to InformationWeek, “CA will announce a partnership with Amazon to provide management capabilities around Amazon’s EC2 utility computing platform, potentially including discovery of software running on EC2 instances, performance monitoring, configuration management, software deployment capabilities and provisioning”.

When someone looks into these two products (and others, soon to follow or alrady out and that I have missed), it will be interesting to see how these Cloud-friendly capabilities relate to the good old capabilities of management products: “software discovery”, “perf monitoring”, “config management”, “software deployment”, “provisioning”. That all sounds pretty familiar. Is it just a matter of pointing the old tools to an EC2 IP address? Is it all new capabilities, done in a new way? Or, more realistically, where does it land between these extrems? Where do you want them to land? It’s not so obvious.

Utility computing comes with an expectation of additional flexibility (now that is obvious). When tweaking IT management tools to address the domain, does one leave “in datacenter” capabilities the same and branch off to do cool things in the new land? Or do you raise the level of flexibility accross the board?

In other words, rather than snickering at them, maybe we should praise IT management vendors for whom the “look, I do Clouds” marketing spiel is just a repackaging of normal IT management features. Because it may mean that they’ve raised the bar on “in datacenter” automation capabilities. These Opsware and BladeLogic acquisitions have to come in somewhere, don’t they?

BTW, both of the announcements above also perpetuate the confusion between providing utility services (CA’s extended SaaS offering, Hyperic’s release of a pre-packaged Hyperic AMI) and the ability to manage Cloud-based systems. It’s all crammed in the same announcement/article because, hey, it’s all Cloud stuff.

Speaking of CA World, if I was there I would go to this session. At least for old time sake, and maybe to get some interesting ideas. Hopefully Don will blog about it after he is done presenting later today.

Pricing is now available for Windows instances on Amazon EC2. More than the technical availability of Windows AMIs, the fact that you get pay your Windows license fee based on usage is a major change. This is where Microsoft’s announcement goes beyond Oracle’s EC2 announcement at Oracle Open World.

But why stop at EC2 instances? If I can do it there, why can’t I do it at home? Considering how rarely my home desktop is booted to Windows, I would love to pay my Windows license in a metered way. It would basically be limited to time spent editing video and participating in family Skype videconference (at least until I manage to get Skype full screen video to work on Ubuntu).

After all, why only Amazon and not other Cloud providers. And when this happens, I think I may become a cloud provider myself. It would be a small-scale operation. One physical CPU (my desktop). And one user (me). I would meter my usage and dutifully pay Microsoft every month based on the number of hours during which I was running Windows.

How much would that be? Well, a Linux Small Standard Image EC2 instance (the closest thing to my aging desktop) costs $0.10 per hour. The Windows version costs $0.125 per hour, so the Windows license on this machine costs 2.5 cents per hour. On a given month, I don’t use it for more than 10 hours (edit/render one DVD plus a few hours on Skype). That’s 25 cents. Does Microsoft take Paypal? Is the Microsoft tax about to get more progressive?

It will be interesting to see how Microsoft manages to be flexible on server OS licensing (where it has plenty of competition) and while keeping its highly profitable (and unfairly front-loaded and restrictive) desktop OS licensing intact.

OVF 1.0.0d is out as a “preliminary standard” so I gave it a quick read over the weekend. Things have not changed much since the “work in progress” document published this summer, which itself wasn’t a big change from the original specification. As I wrote in the review of the “work in progress”, the DMTF tightened the language of the  specification more than it added features.

Since there aren’t too many technical changes (see the end of this post if you’re interested in a few), the interesting discussion is about the marketing of this specification. And boy does it have wings on that front. The level of visibility the specification has received is pretty amazing, especially considering that it doesn’t really do that much technically. But you wouldn’t know it by reading all the announcements about OVF:

• VMWare supports OVF packaging (which version?) with its new VMWare Studio.
• Citrix uses OVF in Kensho to create a platform-agnostic VM management.
• An Open Source “implementation” of OVF has been created. I put “implementation” between quotes because since OVF per se doesn’t do much its implementation is mostly a specialized command line editor for its XML descriptor. It requires a a vendor-specific runtime for deployment/activation. This is not a criticism of the open source project BTW, just a statement of fact about the spec.
• Enomaly lists “OVF format support” on its roadmap for Q1 2009.
• Microsoft support for OVF in products is supposedly “on the board” which doesn’t mean very much but their overall marketing/PR response to OVF has been surprisingly positive for a standard that they don’t control.

I have criticized the DMTF marketing efforts in the past (“give away pens and key chains”) but I must admit that, to the extent that DMTF had a significant role in promoting OVF adoption (in addition to marketing efforts directly from the vendors), it is a very nice marketing success. Well done, and so much for my cynicism. OVF may also have benefited from all the interest in the general topic of virtualization/cloud standards (the “cloud” association is silly, of course, but as we’ve just seen I am not a marketing genius) and the fact that there isn’t much else to talk about on these topics. So by default OVF becomes the name to put on your “standards” banner. Right place at the right time for the vendors behind it.

Speaking of the vendors, I have no insight into the functioning of the OVF working group, but judging by the specification’s foreword VMware is throwing plenty of resources at DMTF: it employs the working group chair and both co-editors, which is pretty atypical in my experience in standards efforts. People are usually sensitive to appearances of one company having disproportionate influence and try to distribute responsibilities around, at least on paper. Add to this VMWare’s recent ramp-up at the DMTF board level. They seem to know what they want. And indeed I can see how the industry leader would want some basic level of standardization, but not too much, which is currently just what OVF offers. We’ll see what’s next in store, if anything.

The specification itself is not marketing-free. According to line 122, “it supports the full range of virtual hard disk formats used for hypervisors today, and it is extensible, which will allow it to accommodate formats that may arise in the future”. Sure, in the same way that my car fully supports passengers of all nationalities (and is extensible enough to transport citizens of yet-to-be created countries - and maybe even other planets, as long as they come with buttocks to sit on). Since OVF doesn’t really do anything with the virtual hard disk formats, it can “support” pretty much any such format.

Speaking of extensibility, OVF clearly tries to have a good story there. Section 7.3 tries to move away from the usual “hey, it’s XML, you can add elements/attributes anywhere” approach towards the definition of new “sections”. This seems a bit drastic. Time will tell if this is visionary or short-sighted. OVF also plans to move towards “an extension model based on the design of the open content model in XML Schema 1.1″. I am not following XSD 1.1 too closely, but it is wise for OVF to not build too much dependency on it at least for now. And it seems to me that an extension model is not something that you plan to “plan [...] to add” but rather something you need to define from the start (sounds like the good old “the next version will add versioning support”, or “no keyboard detected, press F8 to continue”).

But after all this comes what looks to me, from an extensibility perspective, like a big no-no: using (section 8.1) simple strings (e.g. “vmx-4″, “xen-3″) to represent types of virtual systems. You’d think that in 2008 people would have heard about URIs as a way to allow extensibility and prevent name clashes. On further reading, this doesn’t seem to be the fault of OVF as they get this property (vssd:VirtualSystemType) straight out of the politely named DMTF SVP (System Virtualization Profile) specification, itself a preliminary standard. But that’s not much of an excuse because I suspect large overlap of participation between the two groups and in any case you don’t have to take dependencies on something that’s not right (speaking as someone who authored several specs that took a dependency on WS-Addressing, I shouldn’t give lessons). In any case, I am not on top of all virtualization-related work in DMTF but it seems to me that if they are not going to use URIs then someone should step up and maintain a registry of these identifying “virtual system type” strings.

BTW, when left to its own device OVF does a better job. For example, it properly uses URIs to identify the virtual disk format (section 5.2).

One of the few new features is the addition of the ovf:bound attribute on virtual hardware element items (section 8.3) to specify whether the item description represents the normal, minimal or maximal allocation. My heads spins a bit when trying to apply this metadata to the rasd:Limit property (with ovf:bound=”min” the value of the rasd:Limit element would represent the minimal value of the maximum quantity or resources that will be granted, which takes some parsing effort), but I think it more or less squares out.

The final standard should not differ greatly from this version, so at this point we pretty much know what OVF will be technically. The real question is how it will be used and what, if anything, is going to come to complement it.

[UPDATED 2008/10/14: Good timing. OVF-loving Kensho just launched.]

The announcement finally came out. Users can now run supported versions of Oracle Enterprise Linux, 11G Database, Fusion Middleware and Enterprise Manager on Amazon EC2 instances. You can create your own AMI or use any of the pre-packaged AMIs with the above-mentioned products. And you don’t have to purchase new licenses, you can transfer existing ones to run on Amazon’s infrastructure.

A separate but related announcement is the possibility to simply and securely backup your databases on Amazon S3 instead of (or in addition to) on tape. I hope BNY Mellon will take notice.

The Amazon AWS blog has a good overview of the news. Forrester covers it with a focus on data warehousing.

This comes in addition to the existing SaaS offering (“On Demand”) from Oracle and the SaaS platform (for others to provide SaaS on top of Oracle’s software). It is a major milestone for utility computing.

[UPDATED 2008/9/21: This is the home page for the Oracle Cloud Computing Center and this is the FAQ.]

[UPDATED 2008/9/23: More Cloud love, this time with Intel. I have no insight into that partnership.]

Dell took quite a beating last month for (unsuccessfully) trying to trademark the term “Cloud Computing”. This has earned them a reputation as a clown in the Cloud Computing community.

I think it’s unfair. In my experience, the most compelling arguments for Cloud Computing come from Dell. Dell doesn’t make the move to Cloud Computing simply desirable, it makes it indispensable.

How? Not with its “Dell Cloud Computing Solutions” consultants. Not with its XS23 Cloud Server.

With a laptop. The Latitude D420. More specifically, the D420 that I am writing on right now.

I have been using laptops as my primary work machine for over 10 years. This one is by far the worst in terms of stability.

For months, I grappled with undiagnosable crashes. A motherboard replacement fixed those (I think). But the machine still fails to hibernate 20% of the time (sometimes even fresh out of a reboot). And the docking/undocking process is still a roll of the dice. It only works more or less reliably if the laptop is hibernated (but going to hibernation itself is not reliable, see above). If the machine is either turned on or in stand-by, all bets are off. And I am not talking about ending up with a messed up screen resolution. I consider that a successful docking. I am talking about blank screens (laptop and monitor), an unresponsive machine and eventually a hard reboot. By now, the colleagues sitting in the nearby offices must have learned quite a few French swear words.

And please don’t blame Windows XP. It’s not perfect but I’ve had some rock-solid Windows XP laptops, that could go through dozens of hibernate/wake-up cycles and not need a reboot until some OS security patch had to be installed. The NC6400 that I left behind when I quit HP was such an example. More stable than my home Linux laptop.

Anytime my Dell crashes, I risk loosing data in whatever files were open at the time. I’ve become pretty good at rebuilding a corrupted Thunderbird profile and importing the old emails and filters. I’ve learned to appreciate Firefox’s practice to regularly create a backup copy of the bookmarks. I know how to set up auto-save in any application that has the feature. My left hand does the “Ctrl-S” motion on my pillow a hundred times each night.

But above all, I have come to realize how good life will be when all my data, configuration and preferences are in the Cloud. When all my emails, documents, bookmarks, contacts, RSS subscriptions, calendar items are safely removed from this productivity-preventing machine. When recovering from another temperamental bout from this enemy (that I still carry home every day) will only be a matter of logging back onto whatever SaaS application I was using.

Dell has made me a true believer in Cloud Computing.

The first draft of this entry was written (on the afformentioned Linux laptop) during the 13 minutes it takes for the chkdsk.exe process to scan an 80GB hard drive after yet another crash.

There is more and more information coming out about Oslo in anticipation of the Microsoft PDC in October.

David Chappell recorded a video about it last month. More recently Doug Purdy and Don Box each posted a short description of Oslo. Don describes the goal of Oslo as “simplify the process of developing, deploying, and managing software”. But when he lists ancestor technologies to illustrate that “Microsoft has been moving in this direction for over a decade now”, they are all about development, not management: COM type libraries, .NET metadata attributes, XAML. Interesting that neither SDM nor SML gets a mention. Neither did SCA by the way, but I wasn’t really expecting that one… :-)

Maybe the I am the only one looking for a SDM/SML echo here, just because I came to hear of Oslo through the DSI angle. Am I wrong to see Oslo as an enabler for DSI? This eWeek article doesn’t have anything to do with IT management. Reading it, Oslo is all about allowing people to write code through drag and drop. Yawn. And Don Box endorses the article.

Maybe it’s just me (an IT management guy more than a software development guy) but I don’t care so much about how the application model is created. I care a lot more about what it allows you to do in terms of IT management. Please don’t make me pull out the often-quoted figure about the percentage of IT budget spent on operations versus development/licensing. The eWeek piece fails to excite me, but fortunately David Chappell’s video interview is a lot more aligned with my thinking, so I still hold hopes for Oslo as an IT management enabler. Here is my approximate transcript of an example that David provides (at around 4:20) in the video:

“If someone comes to you and says i’ve got this business process and the SLA is not being met, what do you do? You’ve got to trace this through the right business process and the right application that supports that part of the process and find the machine it runs on and maybe look at the workflow that implements it and maybe look at the services that it provides. This involves talking to business analysts, or the IT pros or the architect or the developer, all of whom have their own view of the world, their own tools, their own prospective. The repository provides a common place to store all this stuff, to link it all together, and with a visual editor to have a common tool that lets you actually go through and answer this kind of questions.”

Now you’re talking.

And if Oslo is not the new blood of DSI, then what is? The DSI story is getting dated, SML is fading in our memories and of the three parts that supposedly compose DSI (”virtualized infrastructure, design for operations, and knowledge-driven management”), only virtualization is actually represented on the list of technologies on the DSI home page. Has DSI turned into just allowing System Center to manage a hypervisor? I still hold hopes that the Oslo data is going to spice things up there. It would be good for the industry at large, not just Microsoft.

I won’t be at the PDC but it will be interesting to see what filters out of these sessions. The first session in the list adds management of hybrid application systems (hybrid as in “cloud/on-premise combination” or “software+services” as Microsoft calls it), to the long “can do” list for Oslo. Impressive, if there is some meat behind the abstract. I think this task is often overlooked in discussions around management aspects of Cloud computing (see “the new, interesting thing is going to be the IT infrastructure to manage your usage of utility computing services as well as their interactions with your in-house software” in this previous entry).

Yes, I am reading way too much into session abstracts, but while I am at it I can’t help noticing that there is a lot of SQL and very little XML/XSD/XPath mentioned there. Even though one of the presenters is Gudge, the only person I have ever met who fully understands XSD (actually even he doesn’t, I’ve seen him in the WS-I days have to refer to… his book).

Even though I am sure we’ll be told that SML can be built on top of Oslo, the SQL orientation won’t make that so easy (I want to see how to build XSD+Schematron validation on top of a relational store using Oslo’s drag and drop development tool). And it puts Microsoft on a different architectural direction from IBM, who, as far as I can tell, thinks that the world is a big XML document. Neither is the most appropriate for IT management models. I prefer a graph model and associated graph queries along the lines of SPARQL or CMDBf.

But that’s just late-night idle speculations on my part (aka “blogging”). Let’s see what comes out in October.

[UPDATED 2008/9/10: Interesting timing. Microsoft is joining OMG, home of UML and BPMN. Coming next: a submission of a "new version" of UML and BPMN that happens to contain the extensions and tweaks that Microsoft made to them in the process of implementing Oslo. This, BTW, is the final nail in the SML coffin (SML isn't even mentioned in the press release).]

Around 1995 and 1996, if you knew how to set up an HTTP server on a Solaris box, hand-write a few HTML pages and create a simple CGI script to save the content of a form into a file (extra credit if you remembered to append to the file rather than overwriting it every time), then you were a world-class web designer. At least in my neck of the woods, which wasn’t Silicon Valley at the time. These people were self-trained, of course. I made some side money back then, creating a few web sites with just these limited skills. I am sure there were already people who had really thought about web design and could create useful and attractive sites (rather than simply functional ones). But all twelve of them were busy elsewhere and I would guess that none of them spoke French anyway. They were not my competition in Paris, when talking, for example, to a large French bank who wanted to create a web site to hire college students. My only competition was a bunch of Photoshop clowns whose idea of web design was to create a brochure in Photoshop/Framemaker and make the whole web page one big JPEG file.

Compare this to utility computing (aka clouds) today. Any Linux sysadmin who has, over the last year, made the effort to read and experiment with cloud computing (typically Amazon EC2), to survey available tools and to write a few scripts to tie them together is now an IT rock star, a potential catalyst for operations as a competitive advantage.

Just like self-taught HTML dilettantes didn’t keep control of the web design playground for long, early cloud adopters among sysadmins won’t enjoy they differentiation forever. But I would guess that they do today. Anyone has statistics in terms of valuation for such skills on the job market?

Of course the Photoshop crowd eventually got their Frontpage, Dreamweaver, etc to let them claim that they could create web sites. These tools were pretty bad at first because they tried to make things look familiar to graphic designers (image maps galore!). They slowly got better.

The same thing is likely to happen in utility computing. Traditional IT management tools will soon get cloud features. Like the HTML WYSIWYG tools, they’ll probably tend to be too influenced by current IT management concepts and methods. For example, all the ITIL cheerleaders out there are probably going to bend cloud features to fit ITIL rather than the other way around. Even though utility computing might well invalidate some pretty fundamental assumptions/requirements of parts of ITIL.

The productivity increases created by utility computing are probably large enough that even these tools will provide great value. And they’ll improve. In the same way that the Web was a major enough improvement that even poorly designed web sites were way ahead of the alternatives.

Today, you obviously can’t make a living as an “HTML in notepad” developer. You must either be a real graphic designer and use tools to turn your designs in Web artifacts or be deep in Web technologies. Or both. Similarly, you soon won’t be providing much value if you just know how to start and provision EC2 instances. You’ll need to either be a real IT admin who can manage the utility resources as part of a larger system (like the applications) or be a hard-core utility computing expert who tackles hard problems like optimizing your resource consumption across cloud providers or securing and ensuring the compliance of your distributed IT system.

But for now, the party is raging and the dress code is still pretty lax.

On a recent drive to work, I heard another echo of the Grid world in the context of Cloud computing: I was listening to the Cloud Cafe podcast with Enomaly’s Reuven Cohen when he mentioned (near the 27 minute mark) that they use Ganglia for monitoring their environment.

I am familiar with Ganglia from some HP Labs projects around PlanetLab that I was involved in. Ganglia is used quite a lot for monitoring in the PlanetLab environment.

So Ganglia is one. Is any other project/tool/product coming from the Grid/HPC efforts of the last 10 years now used by the cool Cloud kids? Globus? SmartFrog? Platform? Condor? Others?

A few seconds later in the podcast, Reuven provides this juicy quote: “is the cloud an excuse for bad code”. But that’s a topic for another post.

Grid computing is moulting and, to no surprise, the new skin has “cloud” written all over it.

That’s one way to interpret the announcement today that HP, Intel and Yahoo are going to launch a compute cloud. Seeing Intel and HP work together on this is no surprise. Back at HP I had some involvement with the collaboration between HP Labs and Intel on PlanetLab.

I have only read the Gigaom article and Steve’s, so this post is not an analysis of the announcement. Just a few questions that come to mind. They can be most concisely expressed by trying to understand the difference with Amazon’s EC2. The quotes below all come from the Gigaom article.

“six physical locations” -> Amazon has availability zones, including the choice of three geographies.

“between 1,000 and 4,000 mostly Intel cores” -> According to this well-publicized story, Amazon can deliver 5,000 servers (each linked to at least one physical core) to one customer without breaking a sweat.

“We want, unlike other partnerships including Google and IBM’s where the lower-level stacks are not provided in a open manner to the world, open access to all levels of the hardware” -> The quote seems to conveniently avoid comparison with EC2 which provides a much lower abstraction level: virtual machines with mountable raw block storage devices. How much lower can you go without handing out access cards to physically walk into the datacenter? Access to the BMC on the motherboard? Access to some internal bus? Remote-controlled little robots that will slide cards in and out of a chassis?

“researchers will be able to access the cloud through a proposal process later this year” -> Ec2 offers pay-as-you go, which tends to be a good driver for people to use the infrastructure efficiently. And of course someone can always give researchers a grant in the form of EC2 rent money.

Just to be clear, I am not belittling the announcement because for one thing I haven’t read much about it and for another I probably know many of the HP Labs people involved and they are part of the “mucho sapiens” branch of “homo sapiens”. I know they wouldn’t bother putting this out if it was nothing more than giving researchers some free EC2 time.

But these are the questions I’ll be trying to answer for myself as I read more about this project.

[UPDATED 2008/9/19: Russ Daniels (who was HP Software CTO when I was at HP and is now CTO of Cloud Services Strategy) comments on the announcement.]

A few silly trivia questions for everyone out there who has drunk the Kloud-Aid.

Q) When was the cloudcomputing.com domain registered?

A) February 28, 2007. Yes, less a year and a half ago it could have been yours of 10 bucks. A nice reminder of how quickly the buzzword took over. For comparison, utilitycomputing.com was registered in July 2002 and gridcomputing.com in February 2000. By the way, fogcomputing.com got snapped up a month ago today and is currently parked…

Q) who owns cloudcomputing.com?

A) Dell. Ironically, one of the companies that has the most to loose from it… Of course they don’t see it that way and they redirect that domain to a dell.com page that explains all they have to offer in this area.

Q) Where does the name come from?

A) According to Wikipedia, “the term cloud computing derives from the common depiction in most technology architecture diagrams, of the Internet or IP availability, using an illustration of a cloud”. OK, then are databases now called Cylinder Computing?

Q) How does one make money in Cloud Computing?

A) By registering the domain name and re-selling it at the peak of the hype. CylinderComputing.com is still available…

[UPDATED 2008/8/3: For the record, that last answer was supposed to be a joke. It seemed pretty obvious at the time, but one week later the news comes out that Dell is trying to get a trademark on the term "cloud computing"... More analysis here.]