Let’s first define these terms.

The Functional Model is what the platform does for you. For example, in the case of AWS S3, it means storing objects and making them accessible via HTTP.

The Operational Model is how you consume the platform service. How you request it, how you manage it, how much it costs, basically the total sum of the responsibility you have to accept if you use the features in the Functional Model. In the case of S3, the Operational Model is made of an API/UI to manage it, a bill that comes every month, and a support channel which depends on the contract you bought.

The Operational Model is where the S (“service”) in “PaaS” takes over from the P (“platform”). The Operational Model is not always as glamorous as new runtime features. But it’s what makes Cloud Cloud. If a provider doesn’t offer the specific platform feature your application developers desire, you can work around it. Either by using a slightly-less optimal approach or by building the feature yourself on top of lower-level building blocks (as Netflix did with Cassandra on EC2 before DynamoDB was an option). But if your provider doesn’t offer an Operational Model that supports your processes and business requirements, then you’re getting a hipster’s app server, not a real PaaS. It doesn’t matter how easy it was to put together a proof-of-concept on top of that PaaS if using it in production is playing Russian roulette with your business.

If the Cloud Operational Model is so important, what defines it and what makes a good Operational Model? In short, the Operational Model must be able to integrate with the consumer’s key processes: the business processes, the development processes, the IT processes, the customer support processes, the compliance processes, etc.

To make things more concrete, here are some of the key aspects of the Operational Model.

Deployment / configuration / management

I won’t spend much time on this one, as it’s the most understood aspect. Most Clouds offer both a UI and an API to let you provision and control the artifacts (e.g. VMs, application containers, etc) via which you access the PaaS functional interface. But, while necessary, this API is only a piece of a complete operational interface.

Support

What happens when things go wrong? What support channels do you have access to? Every Cloud provider will show you a list of support options, but what’s really behind these options? And do they have the capability (technical and logistical) to handle all your issues? Do they have deep expertise in all the software components that make up their infrastructure (especially in PaaS) from top to bottom? Do they run their own datacenter or do they themselves rely on a customer support channel for any issue at that level?

SLAs

I personally think discussions around SLAs are overblown (it seems like people try to reduce the entire Cloud Operational Model to a provisioning API plus an SLA, which is comically simplistic). But SLAs are indeed part of the Operational Model.

Infrastructure change management

It’s very nice how, in a PaaS setting, the Cloud provider takes care of all change management tasks (including patching) for the infrastructure. But the fact that your Cloud provider and you agree on this doesn’t neutralize Murphy’s law any more than me wearing Michael Jordan sneakers neutralizes the law of gravity when I (try to) dunk.

In other words, if a patch or update is worth testing in a staging environment if you were to apply it on-premise, what makes you think that it’s less likely to cause a problem if it’s the Cloud provider who rolls it out? Sure, in most cases it will work just fine and you can sing the praise of “NoOps”. Until the day when things go wrong, your users are affected and you’re taken completely off-guard. Good luck debugging that problem, when you don’t even know that an infrastructure change is being rolled out and when it might not even have been rolled out uniformly across all instances of your application.

How is that handled in your provider’s Operational Model? Do you have visibility into the change schedule? Do you have the option to test your application on the new infrastructure or to at least influence in any way how and when the change gets rolled out to your instances?

Note: I’ve covered this in more details before and so has Chris Hoff.

Diagnostic

Developers have assembled a panoply of diagnostic tools (memory/thread analysis, BTM, user experience, logging, tracing…) for the on-premise model. Many of these won’t work in PaaS settings because they require a console on the local machine, or an agent, or a specific port open, or a specific feature enabled in the runtime. But the need doesn’t go away. How does your PaaS Operational Model support that process?

Customer support

You’re a customer of your Cloud, but you have customers of your own and you have to support them. Do you have the tools to react to their issues involving your Cloud-deployed application? Can you link their service requests with the related actions and data exposed via your Cloud’s operational interface?

Security / compliance

Security is part of what a Cloud provider has to worry about. The problem is, it’s a very relative concept. The issue is not what security the Cloud provider needs, it’s what security its customers need. They have requirements. They have mandates. They have regulations and audits. In short, they have their own security processes. The key question, from their perspective, is not whether the provider’s security is “good”, but whether it accommodates their own security process. Which is why security is not a “trust us” black box (I don’t think anyone has coined “NoSec” yet, but it can’t be far behind “NoOps”) but an integral part of the Cloud Operational Model.

Business management

The oft-repeated mantra is that Cloud replaces capital expenses (CapExp) with operational expenses (OpEx). There’s a lot more to it than that, but it surely contributes a lot to OpEx and that needs to be managed. How does the Cloud Operational Model support this? Are buyer-side roles clearly identified (who can create an account, who can deploy a service instance, who can manage a deployed instance, etc) and do they map well to the organizational structure of the consumer organization? Can charges be segmented and attributed to various cost centers? Can quotas be set? Can consumption/cost projections be run?

We all (at least those of us who aren’t accountants) love a great story about how some employee used a credit card to get from the Cloud something that the normal corporate process would not allow (or at too high a cost). These are fun for a while, but it’s not sustainable. This doesn’t mean organizations will not be able to take advantage of the flexibility of Cloud, but they will only be able to do it if the Cloud Operational Model provides the needed support to meet the requirements of internal control processes.

Conclusion

Some of the ways in which the Cloud Operational Model materializes can be unexpected. They can seem old-fashioned. Let’s take Amazon Web Services (AWS) as an example. When they started, ownership of AWS resources was tied to an individual user’s Amazon account. That’s a big Operational Model no-no. They’ve moved past that point. As an illustration of how the Operational Model materializes, here are some of the features that are part of Amazon’s:

• You can Fedex a drive and have Amazon load the data to S3.
• You can optimize your costs for flexible workloads via spot instances.
• The monitoring console (and API) will let you know ahead of time (when possible) which instances need to be rebooted and which will need to be terminated because they run on a soon-to-be-decommissioned server. Now you could argue that it’s a limitation of the AWS platform (lack of live migration) but that’s not the point here. Limitations exists and the role of the Operational Model is to provide the tools to handle them in an acceptable way.
• Amazon has a program to put customers in touch with qualified System Integrators.
• You can use your Amazon support channel for questions related to some 3rd party software (though I don’t know what the depth of that support is).
• To support your security and compliance requirements, AWS support multi-factor authentication and has achieved some certifications and accreditations.
• Instance status checks can help streamline your diagnostic flows.

These Operational Model features don’t generate nearly as much discussion as new Functional Model features (“oh, look, a NoSQL AWS service!”) . That’s OK. The Operational Model doesn’t seek the limelight.

Business applications are involved, in some form, in almost every activity taking place in a company. Those activities take many different forms, from a developer debugging an application to an executive examining operational expenses. The PaaS Operational Model must meet their needs.

Here’s an interesting view of Search Engine Optimization (SEO) from the perspective of a car dealership. I mention this for two reasons:

a) The article is from AutoNews, a site which mostly caters to dealers and others in the car business. Its RSS feed is worth subscribing to in the few months that precede a car purchase. It’s a very different perspective from the consumer-oriented car sites out there.

b) I find it hard to reconcile this article with my experience, as I’ll describe in more details below.

To a large extent, car dealers’ interest in SEO is unsurprising. What business doesn’t care about its Google rank? But, having just bought a Toyota Sienna on-line last week, I have a hard time reconciling dealers’ efforts to get shoppers to their site with how bad the experience is once you get there.

Try and find an email address on the site of a Toyota dealership in Silicon Valley. I dare you. So, forget the email blast. Instead, you have to use stupid “contact us” forms in which you have to copy/paste multiple fields to simply describe what car you’re looking for. And they must know you’re copy/pasting, otherwise would they make the “comments” section so ridiculous small? And the phone number is a “required” field? As if.

I understand that it makes it easier for “lead tracking” software if the transaction starts with one of these forms, but after spending money on SEO do you really want to refuse to talk to customers in the way they prefer? Compare this to going to a dealership in person. They’ll talk to you in their office, they’ll talk to in the showroom, they’ll talk to you in the parking lot in the rain and, gender permitting, they’ll probably talk to you in the bathroom.

I know there are sites (including manufacturer sites) which propose to email local dealers for you, but I don’t know what arrangement they have and I don’t want to initiate a price negotiation in which the vendor already owes a few hundred dollars to a referrer if we strike a deal. I want to initiate direct contact.

At least this year I didn’t have to convince vendors to negotiate the price over email; unlike the first time I bought a car in this manner, three years ago, when over half of the dealers I corresponded with had no interest in going beyond an over-inflated introductory quote, followed by efforts to get me to “come talk at the dealership”. With comments such as “if I give you a price, what tells me you’re not going to take it to another dealership to ask them to beat it?”. Well, that’s the whole point, actually. Nowadays (at least among Toyota dealership in the San Francisco Bay Area) they have “internet sales managers” to do just that.

Once you clear the hurdle of contacting vendors on their web sites, the rest of the interaction is quite painless. “Internet sales managers” deserve their title. In my experience, most of them have no problem doing everything over email and respond in a straightforward way as long as you’re specific in your requests. I never once talked to anyone on the phone. And when I came to take delivery of the car, all had been agreed and my entire visit took one hour, most of it spent doing email while they cleaned the car. I don’t know why anyone would buy a car any other way. The total amount of time I spent on the whole process is less than it would have taken me to go to the closest dealership and negotiate one price.

As a side note, I used TrueCar (under a separate email account, of course) to get an idea of the price and I ended up paying $450 less than the TrueCar proposal. When contacted via their web site, that dealer initially gave me the same offer they had submitted via TrueCar, and we went down from there, based on competing offers from other dealers. I never mentioned the TrueCar offer to them.

Another side note: MSRP is actually very useful. Not as a an indication of the price to pay, of course, but as a checksum on the level of equipment of the car. It doesn’t change between dealers and allows you to ensure that you’re comparing cars with the same equipment level. Of course any decent programmer would scream that it’s a very bad checksum, if only because two options could cost the same, but it worked just fine for my purpose.

I still think that the whole third-party-dealership model is fundamentally broken. The on-line buying process doesn’t fix it, it’s just an added layer that hides some of the issues. As we say in computer science, there’s no problem that can’t be solved by adding another level of indirection. We say this tongue-in-cheek because it’s both true (in the short term) and horribly false (in the long term). The same applies to the US car sales process.

As a side note, now that the family is equipped with an admiral ship I have a 2001 VW Golf Turbo (manual transmission) to sell if anyone in the Bay Area is interested…

• PaaSive Aggressive [link]
• Everything is PaaSible [link]
• You bet your PaaS [link]
• The PaaSibilities are endless [link]

And these are just PaaS puns in blog titles, I’ve probably missed a bunch burried inside other entries.

This must stop. It’s time for the people of the Cloud to stand up and say: ¡No PaaSarán!

Under the title “That Exact Nonsense”, John Gruber posts a quote from Penn Jillette’s book “God, No!: Signs You May Already Be an Atheist” (I left Gruber’s ID in the Amazon URL since he deserve his cut in this context).

There is no god and that’s the simple truth. If every trace of any single religion died out and nothing were passed on, it would never be created exactly that way again. There might be some other nonsense in its place, but not that exact nonsense. If all of science were wiped out, it would still be true and someone would find a way to figure it all out again.

It’s compelling and I personally believe it’s true. But unfortunately it doesn’t prove anything. Clearly, Jillette doesn’t believe that there was a divine revelation for any of the existing religions, but rather that they emanated from human imagination. If that’s the case then yes, the next time around it will come out somewhat differently. But what if there was a divine revelation? What would stop the deity from repeating it if its message got lost?

Jillette only proves that religion is a human invention (not a revelation of a larger truth) if you accept the hypothesis that it is a human invention.

To be fair, Jillette doesn’t claim it’s a proof, but the way the quote is making the rounds (Gruber credits Kottke for the find who in turn credits mlkshk who got it from imgur)  seems to suggest that it’s being heralded as such, rather that as a compelling-sounding tautology.

If you can get past the punny title you’ll read about the following points:

• In traditional (and IaaS) environments, many available application infrastructure features remain rarely used because of the cost (perceived or real) or adding them to the operational environment.
• Most PaaS environments of today don’t even let you make use of these features, at any cost, because of  constraints imposed by PaaS providers for the sake of simplifying and streamlining their operations.
• In the future, PaaS will not only make these available but available at a negligible incremental operational cost.
• Even beyond that, PaaS will make available application services that are, in traditional settings, completely out of scope for the application programmer. Early examples include CDN, DNS and loab balancing services offered, for example, by Amazon. An application developer in most traditional data centers would have to jump through endless hoops if she wanted to control these services within the application. I believe that these network-related services are just the low hanging fruits and many more once-unthinkable infrastructure services will become programmable as part of the application.

PaaS will become less about “hosting” and more about offering application services. In other words, going back to the formula I proposed on Twitter:

Cloud = Hosting + SOA

IaaS is a lot more “hosting” than SOA, PaaS is a lot more “SOA” (application infrastructure services available via APIs) than “hosting”.

You can read the full article for more.

The object of the scorn I’m getting? Nothing to do with the Automator scripts I shared (since I am new to Automator I was hoping to get some feedback/correction/suggestions on that). It’s all about the use case that lead me to Automator in the first place, my refusal to rely on iTunes metadata to organize my music collection.

Look, I’m a software engineer. My employer (Oracle) knows a thing or two about structured data. I work in systems management, which is heavily model-driven. As an architect I really care about consistent modeling and not overloading data fields. I’m also a fan of Semantic Technologies. I know proper metadata is the right way to go. As a system designer, that is, I know it; any software I design is unlikely to rely on naming conventions in file names.

But as a user, I have other priorities.

As a user, my goal is not to ensure that the application can be maintained, supported and evolved. My goal is to protect the data. And I am very dubious of format-specific metadata (and even more of application-specific metadata) in that context, at least for data (like music and photos) that I plan to keep for the long term.

I realize that ID3 metadata is not iTunes specific, but calling it a “standard” that’s “not gonna change” as another commenter, Vega, does is pretty generous (I’m talking as someone who actually worked on standards in the last decade).

Standard or not, here are a few of the reasons why I don’t think format-specific metadata is a good way to organize my heirloom data and why I prefer to rely on directory names (I use the artist name for my music directories, and yyyymmdd-description for photos, as in 20050128-tahoe-ski-trip).

[Side note: as you can see, even though I trust the filesystem more than format-specific metadata I don't even fully trust it and stubbornly avoid spaces in file and especially directory names.]

Some of the pitfalls of format-specific metadata:

• Metadata standards may guarantee that the same fields will be present, but not that they will be interpreted in the same way. As proven by the fact that my MP3 files carry widely inconsistent metadata values depending on their provenance (e.g. “Beattles” vs. “The Beattles” vs “Beattles, The”).
• I can read and edit file names from any programming language. Other forms of metadata may or may not be accessible.
• I don’t have to download/open the file to read the file name. I know exactly which files I want to FTP just by browsing the remote directories.
• I often have other types of files in the same directory. Especially in my photo directories, which usually contain JPEGs but may include some images in raw format or short videos (AVI, MPEG, MOV…). If I drop them in the 20050128-tahoe-ski-trip directory it describes all of them without having to use the right metadata format for each file type.
• File formats die. To keep your data alive, you have to occasionally move from one to the other. Image formats change. Sound formats change. The filename doesn’t have to change (other than, conventionally, the extension). Yes, you still have to convert the actual content but keeping the key metadata in file or directory name makes it one less thing that can get lost in translation.
• Applications have a tendency to muck with metadata without asking you. For example, some image manipulation applications may strip metadata before releasing an image to protect you from accidental disclosure. On the other hand, applications (usually) know better than to muck with directory names without asking.
• You don’t know when you’re veering into application-specific metadata. I see many fields in iTunes which don’t exist in ID3v2 (and even less in ID3v1) and no indication, for the user, of which are part of ID3 (and therefore somewhat safer) and which are iTunes-specific. It’s very easy to get locked in application-specific metadata without realizing it.

[In addition to the issues with format-specific metadata listed above, application-specific metadata has many more, including the fact that the application may (will) disappear and that, usually, the metadata is not attached to the data files. That kind of metadata is a no-starter for long-term data.]

So what do I give up by not using proper metadata?

• I give up richness. But for photo and videos I just care about the date and a short description which fit nicely in the directory name. For music I only care about the artist, which again fits in the directory name (albums are meaningless to me).
• I give up the ability to have the organization of my data reflected in metadata-driven tools (if, like iTunes, they refuse to consider the filesystem structure as meaningful). Or, rather than giving it up, I would say it makes it harder. But either there is a way to automatically transfer the organization reflected in my directories to the right metadata (as I do in the previous post for iTunes) or there isn’t and then I definitely don’t want to have anything to do with software that operates on locked-away metadata.
• I also give up advanced features that use the more exotic metadata fields. But I am not stripping any metadata away. If it happens to all be there in my files, set correctly and consistently, then I can use the feature. If it isn’t (and it usually isn’t) then that piece of metadata hasn’t reached the level of ecosystem maturity that makes it useful to me. I have no interest in manually fixing it and I just ignore it.

I’m not an audiophile. I’m not a photographer. I have simple needs. You may have more advanced use cases which justify the risk of relying on format-specific metadata. To me, the bargain is not worth it.

I’m not saying I’m right. I’m not saying I’m not a grumpy old man. I’m just saying I have my reasons to be a grumpy old man who clutches his filesystem. And we’ll see who, of Mr. D. and me, is crestfallen first.

Here the context: I recently bought my first Mac, a home desktop. That’s where my MP3 reside now, carefully arranged in folders by artist name. I made sure to plop them in the default “Music” folder, to make  iTunes happy. Or so I thought, until I actually started iTunes and saw it attempt to copy every single MP3 file in its own directory. Pretty stupid, but easily fixed via a config setting. The next issue is that, within iTunes, you cannot organize your music based on the directory structure. All it cares about is the various metadata fields. You can’t even display the file name or the file path in the main iTunes window.

Leave it to Apple to create a Unix operating system which hates files.

The obvious solution is to dump iTunes and look for a better music player. But there’s another problem.

Apparently music equipment manufacturers have given up on organizing your digital music and surrendered that function to Apple (several models let you plug an SD card or a USB key, but they don’t even try to give you a decent UI to select the music from these drives). They seem content to just sell amplifiers and speakers, connected to an iPod doc. Strange business strategy, but what do I know. So I ended up having to buy an iPod Classic for my living room, even though I have no intention of ever taking it off the dock.

And the organization in the iPod is driven by the same metadata used by iTunes, so even if I don’t want to use iTunes on the desktop I still have to somehow transfer the organization reflected in my directory structure into Apple’s metadata fields. At least the artist name; I don’t care for albums, albums mean nothing. And of course I’m not going to do this manually over many gigabytes of data.

The easy way would be to write a Python script since apparently some kind souls have written Python modules to manipulate iTunes metadata.

But I am still in my learn-to-use-MacOS phase, so I force myself to use the most MacOS-native solution, as a learning experience. Which took me to Automator and to the following workflow:

OK, I admit it’s not fully MacOS-native, I had to escape to a shell to run a regex; I couldn’t find a corresponding Automator action.

I run it as a service, which can be launched either on a subfolder of “Music” (e.g. “Leonard Cohen”) or on a set of files which are in one of these subfolders. It just picks the name of the subfolder (“Leonard Cohen” in this case) and sets that as the “artist” in the file’s metadata.

Side note: this assumes you Music folder is “/Users/vbp/Music”, you should replace “vbp” with your account user name.

For the record, there is a utility that helps you debug workflows. It’s “/System/Library/CoreServices/pbs”. I started the workflow by making it apply to “iTunes files” and later changed it to work on “files and folders”. And yet it didn’t show up in the service list for folders. Running “pbs -debug” showed that my workflow logged NSSendFileTypes=(“public.audio”); no matter what. Looks like a bug to me, so I just created a new workflow with the right input type from the start and that fixed it.

Not impressed with iTunes, but I got what I needed.

[UPDATE]

I’ve improved it a bit, in two ways. First I’ve generalized the regex so that it can be applied to files in any location and it will pick up the name of the parent folder. Second, I’m now processing files one by one so that they don’t all have to be in the same folder (the previous version grabs the folder name once and applies it to all files, the new version retrieves the folder name for each file). This way, you can just select your Music folder and run this service on it and it will process all the files.

It’s pretty inefficient and the process can take a while if you have lots of files. You may want to add another action at the end (e.g. play a sound or launch the calculator app) just to let you know that it’s done.

In the new version, you need to first create this workflow and save it (as a workflow) to a file:

Then you create this service which references the previous workflow (here I named it “assign artist based on parent folder name”). This service is what you invoke on the folders and files:

I haven’t yet tried running more than one workflow at a time to speed things up. I assume the variables are handled as local variables, not global, but it was too late at night to open this potential can of worms.

If you’re not an existing EM user then those framework upgrades won’t be as visible to you as the feature upgrades and additions. And there are plenty of those as well, from database management to application management and configuration management. The most visible addition is the all-new self-service portal through which an EM-driven private Cloud can be made available. This supports IaaS-level services (individual VMs or assemblies composed of multiple coordinated VMs) and DBaaS services (we’ve also announced and demonstrated upcoming PaaS services). And it’s not just about delivering these services via lifecycle automation, a lot of work has also gone into supporting the business and organizational aspects of delivering services in a private Cloud: quotas, chargeback, cost centers, maintenance plans, etc…

EM Cloud Control is the first Oracle product with the “12c” suffix. You probably guessed it, the “c” stands for “Cloud”. If you consider the central role that IT management software plays in Cloud Computing I think it’s appropriate for EM to lead the way. And there’s a lot more “c” on the way.

Many (short and focused) demo videos are available. For more information, see the product marketing page, the more technical overview of capabilities or the even more technical product documentation. Or you can just download the product (or, for production systems, get it on eDelivery).

If you missed the launch at Open World, EM12c introduction events are taking place all over the world in November and December. They start today, November 3rd, in Athens, Riga and Beijing.

We’re eager to hear back from users about this release. I’ve run into many users blogging about installing EM12c and I’ll keep eye out for their reports after using it for a bit.

There was a lot of interest in the topic in 2009 and 2010. Some heated debates took place during Cloud conferences and a few symposiums were organized to try to coordinate various standard efforts. The DMTF started an “incubator” on the topic. Many companies brought submissions to the table, in various levels of maturity: VMware, Fujitsu, HP, Telefonica, Oracle and RedHat. IBM and Microsoft might also have submitted something, I can’t remember for sure.

The DMTF has been chugging along. The incubator turned into a working group. Unfortunately (but unsurprisingly), it limited itself to the usual suspects (and not all the independent Cloud experts out there) and kept the process confidential. But this week it partially lifted the curtain by publishing two work-in-progress documents.

They can be found at http://dmtf.org/standards/cloud but if you read this after March 2012 they won’t be there anymore, as DMTF likes to “expire” its work-in-progress documents. The two docs are:

• Cloud Infrastructure Management Interface (CIMI) Model and REST Interface Specification, and
• Cloud Infrastructure Management Interface – Common Information Model (CIMI-CIM) Specification

The first one is the interesting one, and the one you should read if you want to see where the DMTF is going. It’s a RESTful specification (at the cost of some contortions, e.g. section 4.2.1.3.1). It supports both JSON and XML (bad idea). It plans to use RelaxNG instead of XSD (good idea). And also CIM/MOF (not a joke, see the second document for proof). The specification is pretty ambitious (it covers not just lifecycle operations but also monitoring and events) and well written, especially for a work in progress (props to Gil Pilz).

I am surprised by how little reaction there has been to this publication considering how hotly debated the topic used to be. Why is that?

A cynic would attribute this to people having given up on DMTF providing a Cloud API that has any chance of wide adoption (the adjoining CIM document sure won’t help reassure DMTF skeptics).

To the contrary, an optimist will see this low-key publication as a sign that the passions have cooled, that the trusted providers of enterprise software are sitting at the same table and forging consensus, and that the industry is happy to defer to them.

More likely, I think people have, by now, enough Cloud experience to understand that standardizing IaaS APIs is a minor part of the problem of interoperability (not to mention the even harder goal of portability). The serialization and plumbing aspects don’t matter much, and if they do to you then there are some good libraries that provide mappings for your favorite language. What matters is the diversity of resources and services exposed by Cloud providers. Those choices strongly shape the design of your application, much more than the choice between JSON and XML for the control API. And nobody is, at the moment, in position to standardize these services.

So congrats to the DMTF Cloud Working Group for the milestone, and please get the API finalized. Hopefully it will at least achieve the goal of narrowing down the plumbing choices to three (AWS, OpenStack and DMTF). But that’s not going to solve the hard problem.

The judge points out that “[t]he term API is slippery”. He refuses to use the term “API” on its own for anything other than “the abstract concept of an application programming interface”. For the sake of greater precision, the judge consistently refers to “API documentation” and “API implementations” when discussing material that was allegedly infringed.

Source

There are many occasions to lament the lack of technical understanding that shines through some legal discussions. But this is a case in which the judge displays a level of precision that we, in the industry, often fail to reach.

Great read, even though that part:

“[Zygna] uses Cloud.com to provide Amazon-compatible (and thus Rightscale-compatible) infrastructure internally, letting it easily move workloads across their own infrastructure and Amazon’s.”

is a bit of a simplification.

While I’m at it, here’s another take on Cloud.com, this time from an OSS license perspective. Namely, the difference between building your business on GPL (like Eucalyptus) or Apache 2 (like the more community-driven open source projects such as OpenStack).

Towards the end, there’s also a nice nod to the Oracle Cloud API:

“DMTF has been receiving other submissions for an API standard. Oracle has made its submission public.  It is based on an earlier Sun proposal, and it is the best API we have yet seen. Furthermore, Oracle has identified a core subset to allow initial early adoption, as well as areas where vendors (including themselves and crucially VMware) may continue to extend to allow differentiation.”

Here’s more on the Oracle Cloud API, including an explanation of the “core/extension” split mentioned above.

But the business [of the rating agencies] started to change in the late 1960s. Instead of charging investors, the rating agencies started to take money from the issuers of the bonds. White blames the shift on the invention of “the high-speed photocopy machine.”

The ratings agencies were afraid, he says, that investors would just pass around rating information for free. So they had to start making their money from the company side. Even this seemed to work pretty well for a long time.

The short term consequences of intellectual piracy may be easy to rationalize: I can imagine that, back in the 60s, stiffing the rich and powerful credit rating agencies by using new technology (the copy machine) to duplicate a report didn’t feel very consequential and maybe even cool. But then there are long term consequences. Like, in this case, a reversal of the money flow which ended up with the issuers themselves paying the rating agencies. A fundamentally unsustainable system which, many years later, gave us credit agencies who coached their customers, the issuers, on how to craft deceptively-rated securities.

We have many examples of too much government intervention (like tax policies that encourage borrowing) setting up the conditions for a crisis, but this appears to be an example of the reverse: a lack of government intervention (to ensure that the IP of credit agencies is respected) being part of what caused a systemic problem to form.

There are similar situations being created today. I cringe when I see advertising (and gathering of personal data) becoming such a prevalent monetization model, by lack of more direct alternatives. I don’t know what form the ensuing crisis will eventually take, but it may be just as bad as the debacle of the credit rating system.

I’m not saying more IP protection is always better. The patent system is an example of the contrary. It’s a difficult balance to achieve. But looking at the money flow is a good gauge of how well the system is working. The more directly the money flows from the real producers to the real consumers, the better and the more sustainable the system is. Today, it isn’t so for credit rating agencies, nor for much of the digital economy.

I posted it as “public” which, Google+ says, means “Visible to anyone (public on the web)”. Except it isn’t. If I go to the link above in another browser (not logged to my Google account) I get nothing but an invitation to join Google+. How AOLy. How Facebooky. How non-Googly.

Maybe I’m doing something wrong. Or there’s a huge bug in Google+. Or I don’t understand what “public on the web” means. In any case, this is not what I need. I want to be able to point people on Twitter who saw my question (and, in some cases, responded to it) to a compilation of answers. Whether they use Google+ or not.

So I copy/pasted the compilation to my blog.

Then I realized that this is obviously what I should have done in the first place:

• It’s truly public.
• It brings activity to my somewhat-neglected blog.
• My blog is about IT management and Cloud, which is the topic at hand, my Google+ stream is about… nothing really so far.
• The terms of use (for me as the writer and for my readers) are mine.
• I can format the way I want (human-readable text that acts as a link as opposed to having to show the URL, what a concept!).
• I know it will be around and available in an open format (you’re probably reading this in an RSS/Atom reader, aren’t you?)
• There is no ad and never will be any.
• I get the HTTP log if I care to see the traffic to the page.
• Commenters can use pseudonyms!

It hurts to admit it, but the thought process (or lack thereof) that led me to initially use Google+ goes along the lines of “I have this Google+ account that I’m not really using and I am sympathetic to Google+ (for reasons explained by James Fallows plus a genuine appreciation of the technical task) and, hey, here is something that could go on it so let’s put it there”. As opposed to what it should have been: “I have this piece of text full of links that I want to share, what would be the best place to do it?”, which screams “blog!” as the answer.

I consider myself generally pretty good at resisting shiny objects, but obviously I still need to work on it. I’m back to my previous opinion on Google+: it’s nice and well-built but right now I don’t really have a use for it.

I used to say “I haven’t found a use for it” but why should I search for one?

Here are the responses I collected:

@wrecks47 challenged the proposition:

• I see the opposite. I observe much more activity in private PaaS rather than public PaaS.

Others seemed to agree and offered these explanations:

@reillyusa and @mfratto think it’s because it’s too hard to build a private PaaS:

• Complex and not too many understandable reference architectures – personified by why Azure appliance is taking so long to appear.
• much harder to build a PaaS in house?

@ryanprociuk and @garnaat think PaaS are very specific (though it’s not clear to me how that explains its lacks of private deployment):

• PaaS may not be as generic as IaaS, specific to a technical solution. IMO
• I think a private PaaS might be very domain-specific. Current PaaS target a narrow range of scale which can be generic.

@somic thinks that in a private setting (presumably without specialized app services) there isn’t much to gain by offering PaaS versus letting people run a container on top of IaaS (though this begs the question why don’t private PaaS provide these services like public PaaS do):

• imho today’s paas, in a private deployment, is just a webapp container – not revolutionary enough to justify a move

@robcheng thinks it’s mostly politics:

• the same agendas that cause companies to embrace private cloud make them suspicious of PaaS (whose jobs/teams become obsolete?)

@cloud_borat‘s interpretation is just that middleware marketers aren’t as savvy as their infrastructure counterparts.

• our expert analyst Igor say that because app server marketing people suck more and not label appserver as private PaaS

Thanks all!

[UPDATED 2011/7/5: This was originally a Google+ post, but it really belongs as a blog post here so I am glad this is where you are reading it. The next post explains why.]

Whether you are a current EM user who wants to make sure you know and use the BSM features or someone just considering EM for that task, this is the book you want.

The full title is Oracle Enterprise Manager Grid Control 11g R1: Business Service Management.

As a bonus feature, and for a limited time only, if you purchase this book over the next 48 hours you get to follow the authors, @ashwinkarkala and @govindars on Twitter at no extra cost! A $2,000 value (at least).

This hasn’t panned out. At least not in that form. Enters AJAX. The XML doc is still there, though it usually wants to be called JSON. The XSLT is now a big pile of JavaScript. That model has many advantages over the XSLT model, the first one being that you don’t have to use XSLT (and I’m talking as someone who actually enjoys XPath). It’s a lot more flexible, you can do small updates and partial page refresh, etc. But does it also maintain the architectural cleanliness of having a data API separated from the rendering logic?

Sometimes. Lori MacVittie describes that model. That’s how the cool kids do it and they make sure to repeat in every sentence that their Web app uses the same API as 3rd party apps. The Twitter web app, for example, is in this category, as Mike Loukides describes. As is Apache Orion (the diagram below comes from the Orion architecture)

That’s one model, and it is conceptually very elegant. One API, many consumers. The Web site of the service provider is just another consumer. Easy versioning. An application management dream (one API to manage, a well-defined set of operations and flows to test, trace and diagnose). From a security perspective, it offers the smallest possible attack surface. Easy interoperability between different applications consuming the same API. All goodies.

And not just theoretical goodies, there are situations where it is the right model.

And yet I am still dubious that it’s going to be the dominant model. Clients of the same service support different interaction models and it’s hard for a single API to work well for all without sprawling out of control (to the point where calling it “one API” becomes a fig leaf). But if you want to keep the API surface small, you might end up with chatty apps. Not to mention the temptation for service providers to give their software special access over those of their partners/competitors (e.g. other Twitter clients).

Take Google+. As of this writing, the web site is up and obviously very AJAX-driven. And yet the API is not available. There maybe non-technical reasons for it, but if the Google+ web site was just another consumer of the API then wouldn’t, by definition, the API already be up?

The decision of whether to expose the interface consumed by your AJAX app as an open API also has ramifications in the implementation strategy. Such an approach pretty much rules out using frameworks that integrate server-side and browser-side development and pushes you towards writing them separately (and thus controlling all the details of how they interact so that you can make sure it happens in a way that’s consumable by 3rd parties). Though the reverse is not true. You may decide that you don’t want that API exposed to 3rd parties and yet still manually define it and keep your server-side and browser-side code at arms length.

If you decide to go the “one REST API for all” route and forgo frameworks that integrate browser code and server code, how much are you leaving on the table? After all, preeminent developers love to sneer at such frameworks. But that’s a short-sighted view.

Some tennis players think of their racket as one tool. Others, who own a stringing machine, think of the frame and the string as two tools, that they expertly combine. Similarly, not all Web developers want to think of their client framework and their server framework as two tools. Using them as one, pre-assembled, tool may not provide the most optimal code, but may still be the optimal use of your development resources.

There’s a bit of Ricardian angle to this. Even if you can produce better JavaScript (by “better” I mean better suited to your need) than the framework, you have a higher Comparative Advantage in developing business logic than JavaScript so you should focus your efforts there and “import” the JavaScript from the framework (which is utterly incompetent in creating business logic).

Just like, in Ricardo’s famous example, Portugal is better off importing its cloth from England (and focusing on producing wine) even though it is, in absolute term, more able to produce cloth than England is.

Which contradicts Matt Raible’s statement that “the only people that like Ajax integrated into their web frameworks are programmers scared of JavaScript (who probably shouldn’t be developing your UI).” His characterization is sometimes correct, but not absolute as he asserts.

I wouldn’t write Google+ with ADF, but it provides benefits to large class of applications, especially internal applications. Where you’re willing to give away some design control for the benefit of faster development and better-tested JavaScript.

Then there is the orthogonal question of whether AJAX technologies are well-suited to a RESTful architecture. You may think it’s obvious, since both are natively designed for the Web. But a wine glass and a steering wheel are both natively designed for the human hand; that doesn’t make them a good pair. Here’s one way to plant doubt in your mind: if AJAX was a natural fit for REST, would we need the atrocity known as the hashbang? Would AJAX applications need to be made crawlable? Reuven Cohen asserts that “AJAX is quite possibly the worst way to consume a RESTful API”, but unfortunately he doesn’t develop the demonstration. Maybe a topic for a future post.

“Because that’s the way it’s done now” was a bad reason to transform perfectly-functional XML-RPC into “message-oriented” SOAP. It also is a bad reason for assuming that your Web application needs to be AJAX-on-REST.

I’ll leave the last word to Stefan Tilkov: “Don’t confuse integration architecture with application architecture.” His talk doesn’t focus on how to build Web UIs, but the main lesson applies. Here’s the video and here are the slides (warning: Flash and PDF, respectively, which is sadly ironic for such a good presentation about Web technology).

As the name suggests, “Yoga” lets you practice some contortions that would strain a run-of-the-mill REST programmer. Basically, you can use a request like

GET /teams/4234.json?selector=:(members:(id,name,birthday)

to retrieve the id, name and birthday of all members of a softball team, rather than having to retrieve the team roaster and then do a GET on each and every team member to retrieve their name and birthday (and lots of other information you don’t care about).

Where have I seen this before? That use case came up over and over again when we were using SOAP Web services for resource management. I have personally crafted support for it a few times. Using this blog to support my memory, here is the list of SOAP-related management efforts listed in the “post-mortem on the previous IT management revolution”:

WSMF, WS-Manageability, WSDM, OGSI, WSRF, WS-Management, WS-ResourceTransfer, WSRA, WS-ResourceCatalog, CMDBf

Each one of them supports this “partial access” use case: WS-Management has :

WSMF, WS-Manageability, WSDM, OGSI, WSRF, WS-Management, WS-ResourceTransfer, WSRA, WS-ResourceCatalog, CMDBf

Each one of them supports this “partial access” use case: WS-Management has SelectorSet, WSRF has ResourceProperties, CMDBf has ContentSelector, WSRA has Fragments, etc.

Years ago, I also created the XMLFrag SOAP header to attack a more general version of this problem. There may be something to salvage in all this for people willing to break REST orthodoxy (with the full knowledge of what they gain and what they loose).

I’m not being sarcastic when I ask “where have I seen this before”. The problem hasn’t gone away just because we failed to solve it in a pragmatic way with SOAP. If the industry is moving towards HTTP+JSON then we’ll need to solve it again on that ground and it’s no surprise if the solution looks similar.

I have a sense of what’s coming next. XPath-for-JSON-over-the-wire. See, getting individual properties is nice, but sometimes you want more. You want to select only the members of the team who are above 14 years old. Or you just want to count these members rather than retrieve specific information about them individually. Or you just want a list of all the cities they live in. Etc.

But even though we want this, I am not convinced (anymore) that we need it.

What I know we need is better support for graph queries. Kingsley Idehen once provided a good explanation of why that is and how SPARQL and XML query languages (or now JSON query languages) complement one another (wouldn’t that be a nice trifecta: RDF/OWL’s precise modeling, JSON’s friendly syntax and SPARQL’s graph support – but I digress).

Going back to partial resource access, the last feature is the biggie: a fine-grained mechanism to update resource properties. That one is extra-hard.

For example, the Twitter API ToS:

“The Rules will evolve along with our ecosystem as developers continue to innovate and find new, creative ways to use the Twitter API, so please check back periodically to see the most current version.”

and more specifically

“Twitter may update or modify the Twitter API, Rules, and other terms and conditions, including the Display Guidelines, from time to time its sole discretion by posting the changes on this site or by otherwise notifying you (such notice may be via email). You acknowledge that these updates and modifications may adversely affect how your Service accesses or communicates with the Twitter API. If any change is unacceptable to you, your only recourse is to terminate this agreement by ceasing all use of the Twitter API and Twitter Content. Your continued access or use of the Twitter API or any Twitter Content will constitute binding acceptance of the change.”

Changes in the terms of service can have a major effect on consuming application, going as far as making the whole purpose of the application impossible. For example, any Twitter app is a the mercy of the next corporate strategy change decided at Twitter HQ. That can happen without any change whatsoever to the API from a technical perspective.

If building your business on a proprietary API is sharecropping, then changes in ToS are the “droit de cuissage” (a.k.a “droit du seigneur“) that comes with it. Be ready for it to be exercised at any point.

Even in the cases where the ToS contain a guarantee over time, there is often enough language to allow the provider to wiggle out of it. Case in point, the recent decommission of the Google Translate API.

In the API ToS, Google includes a generous three years deprecation window:

“If Google in its discretion chooses to cease providing the current version of the Service whether through discontinuation of the Service or by upgrading the Service to a newer version, the current version of the Service will be deprecated and become a Deprecated Version of the Service. Google will issue an announcement if the current version of the Service will be deprecated. For a period of 3 years after an announcement (the “Deprecation Period”), Google will use commercially reasonable efforts to continue to operate the Deprecated Version of the Service and to respond to problems with the Deprecated Version of the Service deemed by Google in its discretion to be critical. During the Deprecation Period, no new features will be added to the Deprecated Version of the Service.”

But further down, Google reserves the right to do away with the Deprecation Period for several reasons, among which:

“- providing the Deprecated Version of the Service could create a substantial economic burden on Google as determined by Google in its reasonable good faith judgment; or
– providing the Deprecated Version of the Service could create a security risk or material technical burden upon Google as determined by Google in its reasonable good faith judgment.”

A “substantial economic burden” or a “material technical burden”. I am not a lawyer, but good luck going to court against Google arguing that maintaining the API doesn’t provide any such burden if Google claims it does (if I was a Google lawyer, I’d ask “if there’s no such burden, why don’t you provide it yourself?”).

And that’s exactly what Google is claiming in the case of the Translate API. There’s a reason why the words “substantial economic burden” are included in this statement at the very top of the home page for the Google Translate API.

“Important: The Google Translate API has been officially deprecated as of May 26, 2011. Due to the substantial economic burden caused by extensive abuse, the number of requests you may make per day will be limited and the API will be shut off completely on December 1, 2011. For website translations, we encourage you to use the Google Translate Element.”

[For more information about Google's translation services and what might really be behind this change, I recommend reading this interesting analysis. Whether or not the author is guessing right, it's an interesting perspective on how the marriage between Search and Advertising turns into a love triangle (with Translation as the third party) when you take a worldwide perspective.]

Protection against such changes (changes in ToS or the API going away entirely) would be a nice value-add for the companies that offer services around managing API usage. I vaguely remember an earlier announcement about Apigee providing a way to invoke the Twitter API that was sheltered from Twitter’s throttling (presumably via some agreement between Apigee and Twitter). Maybe Apigee can also negotiate long-term bidirectional ToS agreements with API providers on behalf of its customers. If I was building a business on top of a third-party service like Twitter, I’d probably value that kind of guarantee even more than the cool technical tools Apigee provides around API usage.

What’s more important in an API? Whether it’s REST or SOAP or whether you can count on using it for some time in the future?

Here are the highlights of his verdict, with some comments.

“Supporting both JSON and XML [is good]“

I disagree: Two versions of a protocol is one too many (the post behind this link doesn’t specifically discuss the JSON/XML dichotomy but its logic applies to that situation, as Tim Bray pointed out in a comment).

“REST is good, SOAP is bad”

Not necessarily true for all integration projects, but in the context of Cloud APIs, I agree. As long as it’s “pragmatic REST”, not the kind that involves silly contortions to please the REST police.

“Meaningful error messages help a lot”

True and yet rarely done properly.

“Providing solid API documentation reduces my need for your help”

Goes without saying (for a good laugh, check out the commenter on George’s blog entry who wrote that “if you document an API, you API immediately ceases to have anything to do with REST” which I want to believe was meant as a joke but appears written in earnest).

“Map your API model to the way your data is consumed, not your data/object model”

Very important. This is a core part of Humble Architecture.

“Using OAuth authentication doesn’t map well for system-to-system interaction”

Agreed.

“Throttling is a terrible thing to do”

I don’t agree with that sweeping statement, but when George expands on this thought what he really seems to mean is more along the lines of “if you’re going to throttle, do it smartly and responsibly”, which I can’t disagree with.

“And while we’re at it, chatty APIs suck”

Yes. And one of the main causes of API chattiness is fear of angering the REST gods by violating the sacred ritual. Either ignore that fear or, if you can’t, hire an expensive REST consultant to rationalize a less-chatty design with some media-type black magic and REST-bless it.

Finally George ends by listing three “ugly” aspects of bad APIs (“returning HTML in your response body”, “failing to realize that a 4xx error means I messed up and a 5xx means you messed up” and “side-effects to 500 errors are evil”) which I agree on but I see those as a continuation of the earlier point about paying attention to the error messages you return (because that’s what the developers who invoke your API will be staring at most of the time, even if they represents only 0.01% of the messages you return).

What’s most interesting is what’s NOT in George’s list. No nit-picking about REST purity. That tells you something about what matters to implementers.

If I haven’t yet exhausted my quota of self-referential links, you can read REST in practice for IT and Cloud management for more on the topic.

We can take “road” literally in this case since Toyota has teamed up with Salesforce.com to “build Toyota Friend social network for Toyota customers and their cars“.

If you’re tired of “I am getting a fat-free decaf latte at Starbucks” FourSquare messages, wait until you start receiving “my car is getting a lead-free 95-octane pure arabica gas refill at Chevron”. That’s because Toyota owners will get to “choose to extend their communication to family, friends, and others through public social networks such as Twitter and Facebook“.

Leaving “family and friends” aside (they will beg you to), the main goal of this social network is to connect “Toyota customers with their cars, their dealership, and with Toyota”. And what for purpose? The press release has an example:

For example, if an EV or PHV is running low on battery power, Toyota Friend would notify the driver to re-charge in the form of a “tweet”-like alert.

That’s pretty handy, but every car I’ve ever owned has sent me a “tweet-like” alert in the form of a light on the dashboard when I got low on fuel.

Toyota’s partner, Salesforce, also shares its excitement about this (they bring the Cloud angle), and offers another example of its benefits:

Would you like to know if your dealer’s service department has a big empty space on its calendar tomorrow morning, and is willing to offer you a sizable discount on routine service if you’ll bring the car in then instead of waiting another 100 miles?

Ten years ago, the fancy way to justify spamming people was to say that you offered “personalization”. Look at this old advertisement (which lists Toyota as a customer) about how “personalization” is the way to better connect with customers and get them to buy more. Today, we’ve replaced “personalization” with “social media” but it’s the exact same value proposition to the company (coupled with a shiny new way to feed it to its customers).

BTW, the company behind the advertisement? Broadvision. Remember Broadvision? Internet bubble darling, its share price hit over $20,000 (split-adjusted to today). According to the ad above, it was at the time “the world’s second leading e-commerce vendor in terms of licensing revenues, just behind Netscape and ahead of Oracle, IBM, and even Microsoft” and “the Internet commerce firm listed in Bloomberg’s Top 100 Stocks”. Today, it’s considered a Micro-cap stock. Which reminds me, I still haven’t gotten around to buying some LinkedIn…

Notice who’s missing from the list of people you’ll connect to using Toyota’s social network? Independent repair shops and owners forums (outside Toyota). Now, if this social network was used to let me and third-party shops retrieve all diagnostic information about my car and all related knowledge from Toyota and online forums that would be valuable. But that’s the last thing on earth Toyota wants.

A while ago, a strange-looking icon lit up on the dashboard of my Prius. Looking at it, I had no idea what it meant. A Web search (which did not land on Toyota’s site of course) told me it indicated low tire pressure (I had a slow leak). Even then, I had no idea which tire it was. Now at that point it’s probably a good idea to check all four of them anyway, but you’d think that with two LCD screens available in the car they’d have a way to show you precise and accurate messages rather than cryptic icons. It’s pretty clear that the whole thing is designed with the one and only goal of making you go to your friendly Toyota dealership.

Which is why, without having seen this “Toyota Friend” network in action, I am pretty sure I know it will be just another way to spam me and try to scare me away from bringing my car anywhere but to Toyota.

Dear Toyota, I don’t want “social”, I want “open”.

In the meantime, and since you care about my family, please fix the problem that is infuriating my Japanese-American father in law: that the voice recognition in his Japan-made car doesn’t understand his accented English. Thanks.