Will Virtualization Kill the Cloud?

I know there is a lot of contention over what exactly cloud computing means. Some people use the metaphor as “compute as a utility” (like electricity) – which seems to be more of a long term “grand challenge” to me.  I found a short description that I believe is right on the mark by James Urquhart - “Cloud computing is an application-centric operations model.” So for IT applications become king – and the whole of the IT will be focused around serving applications and application users. Even though sounds almost intuitive to most business folks (I mean what else is IT except a way to get applications to users?) it isn’t how most IT departments operate. In most IT departments – infrastructure is king, not applications. You want to deploy a new application or you need more resources for an existing application – well then wait a few months for the infrastructure folks to requisition, provision, integrate and provide you those resources. The cloud will play havoc with that model. There is a reason for this mismatch in paradigms. It is because for the infrastructure folks, cloud or not,  compute and storage resources are not flexible or infinite – even though they may appear that way to the application folks. Infrastructure is a physical resource – and therefore not unlimited.

It clear that virtualization is the mechanism that most cloud providers will use to try to create the illusion of “unlimited available resources” at the application layer. Virtualization isn’t new to the data center (mainframes have been doing it forever, and VMWare has been around for quite a few years now). What is new in the public cloud infrastructure – the organization using the cloud is completely blind to physical infrastructure and its topology – all you get to see is your VMs and the stack above those VM.

That blindness means that you don’t know if your apps are running on machine with 10 other apps, or if your VM has just been migrated to another physical server. Or maybe your cloud provider has skimped a bit on infrastructure, and now physical machines need to host a few more virtual machines to meet the needs of some peak period. In a perfect world that wouldn’t matter – but the world isn’t perfect. Your apps will be affected by their physical neighborhood – for example take the “noisy neighbors” problem that I mentioned in “Noisy Neighbors, Amazon Cloud and the Mainframe”. All of a sudden, through no fault of your own, your production applications may start acting erratically. So now you’ll need to understand that your performance and SLA problems may be caused by things that you can’t see, and can’t access – and will never be able to access.

Going back to the “compute as a utility” metaphor – virtualization makes the cloud is very different than say, an electric utility. You wouldn’t be very happy if your refrigerator was working 5 degrees warmer because the guy down the block turned on his air-conditioner – but that is what can happen when virtualization is used in the cloud. Just like with the refrigerator – you won’t know about the problem until it it is too late – the food spoils, irate users are on the phone.

So as the cloud matures – the issues brought about by mass virtualization will need to be addressed – or we may find that virtualization killed the cloud.

Not Unstructured, Not Unpredictable, Not Ad-hoc Processes – Simply Knowledge Processes

There is an interesting converstaion going on at the Adaptive Case Management group in LinkedIn. It made me notice that people continually struggle with how to describe the kinds of processes covered by adaptive case management – people use the terms unstructured, unpredictable and ad-hoc interchangeably (I too am guilty of that). The problem is that all those terms either mean something is missing or wrong with the process, and insinuate that if only we would work harder or smarter we could change any process to a more structured, predictable, well-defined, recurring process – which just isn’t true for knowledge processes. Do we as a community really believe that knowledge work will be mostly defined by predefined structured process? I don’t think so. I think we should call these types of process “knowledge processes” (which puts them in a positive light) rather than unstructured (or any of the other related terms) processes.

The key issue in morphing technology support from structured processes to knowledge processes (interesting how the words structured vs. knowledge sound when said aloud -try it) is changing the mindset from one of control to one of visibility, guidance and tracking. Knowledge processes demand inversion of control - it isn’t the process (and its model) which controls the participants, but rather the participants which control process and its flow. Optimizing a knowledge process isn’t about counting steps – but rather optimizing outcomes and appropriately leveraging skills. Managing a knowledge process isn’t about control, it is about providing guidance about possible next steps, ensuring appropriate levels of visibility into process execution, collaboration between process participants and tracking process execution through its steps.

Both the vendor and analyst ACM communities are mistakenly worried about exactly what technical features need to be included or excluded in an ACM tool. No matter what we all think about our own aproaches – there is no single right answer. I think that is main problem of ACM  - both the vendor and analyst communities immediately drill down to the technical features of the tooling – losing the bigger business picture.

Preventing Failure vs Fixing Failure

There has been a lot of discussion on the value of failure in the ACM\BPM community in last few weeks (Failure is Essential to Knowledge Work,  The Value of Failure, Preventable Faillure, Unavoidable Failure, Intelligent Failure).  Of course failure is part of any process (i.e. for some reason the process didn’t achieve a desired result) , though sometimes we use the word exception in the context of a process.

One of the key reasons companies deploy a BPM suite is to prevent failure. This is a major selling point for many BPM solutions. A key goal of a BPM suite is to enable the deployment of process driven solutions that prevent a deployed process from failing. As everyone knows – preventing failure is a lot cheaper than fixing it, so any technology that can help prevent failure is valuable. But is that really true in every context?

When deploying a BPM solution the question you need to ask is whether your processes are well defined enough and correct enough that you can really focus on preventing failure (and don’t forget in most cases  preventing failure is equivalent to limiting options). If that isn’t true (which is the case more often then people think) – then you should focus on the ability to fix failure fast, and correctly. Or said another way – the focus needs to be on “first fault failure resolution” capability rather than failure prevention capability.

I found this document from Netflix which I think is very interesting given the public failures they have gone through lately.  It has a lot of insight from a rapidly growing company about how they think about process (starting on slide 44), though the derogatory term bureaucracy is used. Here are some interesting process related quotes from the presentation:

• Process brings seductively strong  near term outcome
• “Good” process helps talented people get more done
• “Bad” process tries to prevent recoverable mistakes
• Embrace Context – Not Control
• In a creative-inventive market, not a safety-critical market like medicine or nuclear power. You may have heard preventing error is cheaper than fixing it — Yes, in manufacturing or medicine…but not so in creative environments.

So in summary -  where you want to play it safe deploy a process solution focused on managing structured processes, if you need agility (and are willing to accept its associated risk) then you should focus on “first fault problem resolution” for your unstructured processes – rather than try to structure them to prevent failure.

On the CIO – CEO Gap

I read an interesting article on the gap between CIO and CEO technology priorities. Looking at the Gartner 2012 technology priorities (and using that as a proxy as what CIOs think is important) the CIO’s list revolves around Tablets, Mobile, Social, BI and Cloud. On the other hand the CEOs list is:  ERP, CRM, Specific business-line applications, E-commerce expansion, General IT modernization,  IT infrastructure improvements, Business mobility as it relates to major platforms, Business intelligence, Supply chain management and Security – interesting that process doesn’t show on either list, but I’ll leave for later.

What can I tell you?  CEO’s are right – they are essentially saying that a lot of previous technology initiatives aren’t finished yet, and they still need to maximize their value to the business. CEOs are also saying that technology isn’t interesting unless it serves a business purpose. The CEO’s list focuses on technology initiatives that actually affect the business or as I like to think about it can either optimize and streamline current business delivery (increase the bottom line) or generate new business (increase the top line). Most of the CEO’s focus is on ITs ability to increase the bottom line, not the top, which is the gist of how CEOs view CIOs – as the owner and manager of the infrastructure used to make the business machinery run – not as a partner on the business side, and certainly not as a seat of business innovation. Many CIO’s balk at being pigeonholed in this way, they would like to consider themselves full fledged partners on the business side and maybe even as a candidate for CEO.

I think that the technology lists really highlight the difference between a CEO and CIO. The CEO focuses on things like ERP which is “technology in the large” – a number of technologies applied as a large business transforming initiative. The CIO list is “technology in the small” specific technologies that have unclear impact on the business as a whole.

CIOs need to think of technology in the large – and they actually have the technical ability to translate it into the different small technologies needed to make the initiative work. CIOs need to view technology as a tool for business, not an end in itself. They shouldn’t focus on ”cloud computing” but rather the business benefits of “agility” – and then break that down into what is really needed by to enable agility (and maybe the cloud fits).

Here is my list of possible CIO technology initiatives, each has a set of related technologies – but I am sure there are many others I have left out (and hopefully others will point them out to me):

1. Process management – since IT touches almost every business process, the CIO is really the only person that could understand how the business actually runs.  By taking a process oriented view of how IT supports the business a CIO could create real value for the company.  There are quite a few process management related technologies that could be relevant, and process management should certainly be on the short list of initiatives  for any CIO that wants to be CEO. I am surprised it didn’t show up on the technology list at all – though it did on the CEO’s list through ERP, CRM, applications and supply chain management.

2. Running IT as a Business – I am always surprised about how IT is actually run - IT could be a showcase of how information technology can be used to manage business, but it usually is not (and quite often the opposite). Eat your own dogfood – need I say more?

3. Leveraging Data for Business Value - the good news is this is actually on both lists. The bad news is the IT itself doesn’t do such a good job of leveraging their own data for their own needs. Again – shouldn’t IT be a showcase of how data can help make a business run better? Shouldn’t they be doing it themselves for their own business?

4. Application agility – applications are what business cares about – not how applications are delivered. The good news is that some of the technologies on the CIO’s list could be used to target appplication agility – cloud, app orientation, mobile – but if they are viewed as separate, distinct activities they probably won’t end up having a profound impact on the business. The point here is that the technology doesn’t matter from a business perspective – but applications do. How many innovative applications that help the business has IT come up with lately?

Are BPM suites another 4GL (fourth generation programming language)

Lately I have been spending more time with various IT issues (especially around application performance management) than with business process management (BPM) or adaptive case management (ACM). What I think surprised me most is how little BPM is used by most development and IT shops for their own use. Even when a truly structured IT process is being implemented (like deploying an application into production) – the tools are never based on BPM suites. The only time I see BPM suites being used is when there is an IT management decision to take a more process oriented approach to certain business applications. In that case a BPM team is created and they implement certain processes – it never seems to leak much into the broader IT domain. It also seems to be used mostly in the context of business application built by enterprise IT departments.

So where does that leave BPM suites? - they aren’t used by most developers, and aren’t used by business people. So in many ways it resembles a 4GL (a fourth generation languag,e for anyone old enough to remember) for implementing structured business processes that don’t have a packaged application available. Just like 4GLs in their day it provides value for a nice sized niche – but still a niche.

What does that mean for the future of BPM suites?

BPM suites could try to branch out and try to encompass unstructured processes – but that would mean essentially building a different tool as Max Pucher continually points out in his blog and Neil Ward-Dutton shows in his presentations on ACM vs BPM.  This can work in a suite context (as Keith Swenson points out in his blog) and I believe that this is where many BPM suites are headed – but I doubt if they’ll make it. It is a lot more complex then just adding some “social features” to BPMS. Most vendors don’t understand that, and neither do many analysts – just take a look at how much focus Forrestor puts on the structured part of case management (and how little on the unstructured part) -  in their dynamic case management wave. As I have said before we’ll know BPM vendors have nailed it when we see knowledge workers using their BPM applications on a daily basis instead of email.

A second direction would be to provide more value to participants in the niche – or expand to the folks who can benefit from data derived from the niche (like BI capabilites). I think all BPM suites will expand in this direction – but I question whether that will be enough to keep them from going the same path as 4GL languages – essentially a niche business that never breaks out into the mainstream -either from a IT or business direction.

A third direction would be for BPM suites to embrace their 4GLness and focus providing more value to developers so that BPM suites could be used by a broader set of developers, while evolving into general purpose business application developments suites – but I don’t see any BPM vendors going in that direction.

If Agile Software Is A Cop-Out; Is BPM the Next Studio for Software Development?

Continuing my comments on Mike Gualtieri’s post on the state of software development (which seems to have hit on a real issue, based on the number and quanitity of comments it generated), Mike states that there are 4 pillars needed for a methodology for creating a new paradigm for software development:

• Parallel. Development teams must have the confidence and insight to do parallel streams of development.
• Immersive. Software is a means to an end. That end is to deliver applications that exhibit the seven qualities of great software: user experience, availability, performance, scalability, adaptability, security, and economy.
• Software. That’s what we develop. That’s what we deliver. However, the future software development is experience.
• Studio. Software development is not pure coding, engineering, architecture, management, or design. It is cross-disciplinary. Better yet, it is its own discipline.

These are all good, important points but they ignore the business perspective! I have been a user experience advocate forever – since being the user experience advocate for IBM Research in the late 90s. User experience is a critical perspective for developers to understand, and critical for the success of any software. But I have learned it isn’t enough – software development needs to take into account two more critical business aspects:

• Business Value. Software must deliver business value, and that isn’t always completely aligned with user experience or any of the immersive points Mike lists. A application can have a great user experience but bring no value to the business, which means it will either be killed or die from lack of attention. In most cases business value needs to come first, and foremost.
• Platform. Software (even what ends up as applications software) is either a platform itself, or built on (and used to enrich) a platform. This is both a business and technical decision, and can greatly affect the long term success of any software. See a great rant on the subject here.

So how do BPM platforms stack up as supporting the pillars of software development?  Theoretically they stack up pretty well.

• Parallel – the modeling process is a bottleneck, but once completed things can be implemented in parallel
• Immersive - the BPM and its model driven development platform should make it much easier to obtain the immersive aspects too.
• Software – most BPM platforms put no focus on user experience. I have written about this before, but it a real failing of most BPM suites.
• Studio – certainly BPM suites are built to support multi-disciplinary teams (though with great differences in the level of support for different audiences)
• Business value – BPM suites should get high ratings here – since the theory is that the is driven by “the business”
• Platform – A BPMS is by definition a platform, and theoretically every application built on the BPMS could enhance it

The problem is the distance between theory and practice. Most BPM suites try too hard to make themselves of value to the business side, and miss a lot of what is needed on the high-end development side. I really don’t know of any BPM suite that focuses solely on developers and their needs, or positions itself as a better development environment for business applications. BPM suites are sort of a camel, while developers are looking for a better horse. So BPM suites could be part of the answer for the problems in software development, but in practice they aren’t.

Aren’t Software Developers the Archetype Knowledge Worker?

I haven’t been posting much on case management lately, and have been focusing on other topics. One post I read the other day about software development really got me thinking about unstructured processes again – I thank Mike Gualtieri for his great post on “Agile Software Is A Cop-Out, Here’s What’s Next“.

Mike makes an interesting point about agile software development methodologies. Whenever I hear about agile methodologies I get confused – since top software developers and designers always used agile as part of their bag of tricks – just never codified it to death. That is what I see as the main failing of agile it started as a way to spread knowledge aen experience, but is now trying to morph into a standardized methodology. It took various apparent truths about software development and then tried to build a set of “rigorous” processes around them. The problem is that “agile as process” took on a life of its own and lost the original purpose. From my perspective that is why a manifesto works much better than a methodology/process – but once a manifesto is hijacked by consultants and vendors – they immediately make it a “process”. I actually think that is what is happening in the case management\bpm debate too – what needs to remain as a flexible methodology supported by tools, is slowly being killed so that it can be standardized.

What I don’t get is that software development is a shining example of what people in the process community call knowledge work. As far as I can tell, a lot of the latest thinking around managing knowledge work is being done by the business process and case management communities. Even though most of the people talking about tools for BPM and case management are software developers themselves – the tools and methodologies advocated by BPM\Case management are not being adopted by the software development and operations community themselves. I see this as vendors not eating their own dog-food (or maybe not drinking their own Kool-Aid) and I have been posting about it for a while – see my last post on this and Scott’s great rebuttal here.

So from my perspective a lot of the thinking around knowledge work is for techies and analysts talking about “other people’s knowledge work”, not for the work they themselves do. I don’t understand the wide gulf between knowledge work as it is done by software developers, consultants and analysts – or any other “specialty” knowledge work,  and the work done by “standard” business process knowledge professionals and executives (who seem to called case workers by the community?) -

1. All software development in commercial enterprise is being done in service of a business process – therefore all software development should also be considered part of business process management. It is a sub-process of the overall business “aha to ka-ching” process – from an idea to increase\enhance business to the implementation and deployment of that idea through software.

2. Software developers are knowledge workers. Their macro processes are the same as those for any knowledge worker. Like any knowledge worker they also use specialty tools,  tailored for their specific domain. Therefore a BPM tool that isn’t used to support software development as a process (e.g. Waterfall and ITIL, or Agile and Iterative methodologies) while connecting to domain specific tooling is a cop out, a way for vendors to say “use this tool for all other knowledge processes, but of course I won’t use it for mine”.

3. Analysts and consultants are knowledge workers- if they aren’t using a tool for managing their own knowledge work, then they too suffer from the ”all knowledge work except mine” syndrome. If all they use are standard productivity tools for their work (e.g. email, wikis, blogs, twitter, word processing, spreadsheets and presentations) – then I would ask them what is keeping them and their organizations from adopting the tools they say your organization should adopt.

Selling to the Enterprise as a Startup

In my last post I gave some rules for Enterprise startups. Once started the technology (no matter how complex) is the easy part – the hard part is getting noticed,  then closing the deals and getting the sale. This is where most startups (not only enterprise
startups) fail.  Of course some failure modes are simple – the technology just doesn’t work, the startup doesn’t have a
large enough sales pipeline, or the average selling price is too low to justify an enterprise sales model. There are a number of “failure” modes that are more insidious – they start by looking like success, but then turn into a quagmire.

These are more problematic since they tend to happen after quite a bit of money has been invested and management commitments have been made to the board. Once commitments are regularly missed the board starts losing trust in the executive management – and the board\investors feel the need for some drastic action. Here are some examples of those kinds of failure modes:

1. Early elephants skew the numbers – the startup starts selling and lands a couple of elephants – $1M deals that look like they represent a replicable sales process. Everyone pats themselves on the back, and spent lots of money on scaling sales and marketing (and maybe even development) to make sure no opportunity falls through because of lack of resources. The sales forecasts spiral upward since the additional expenditures need to be justified (and based on recent sales history the numbers make sense).  Then the well dries up – it turns that those sales aren’t repeatable and the company can’t make its numbers.
2. Deals start easily – but close verrrry slowly. Even under the best circumstances enterprise software deals take a long time to close so people are willing to wait – but sometimes it starts dragging out into forever – a deal taking 6 months to close can be OK, but over a year is a stretch. The problem is that until you know those deals aren’t closing, at best 6 months have passed – forever in the life of a startup.
3. Personnel changes at the customer. Even if the startup knows their customer and why they buy (I am always startled by how many don’t really understand that) – reorgs can make your sponsor disappear. So you’ve invested 3 months in selling to someone that is no longer relevant – and it isn’t clear that now anyone cares.

There are ways around each of these scenarios – but the key is to be vigilant and as Andrew Grove puts it – paranoid. The problem is that in many cases this goes against the optimistic outlooks that many entrepreneurs naturally have…

The 10 Commandments of Enterprise Startups

There was an interesting article on enterprise startups in CIO.com – Enterprise Startups: A Long Road Ahead. I started my career at IBM, but have been doing startups for a while now – and by far most have been enterprise software startups (Actionbase, ConicIT, Correlsense, Itemfield). eXeedTechnology and Dapper were the only “non-enterprise” software startups of the bunch. What was said by the jury in the article (or at least what how I read it) really resonated with me, and the experience I have with enterprise software startups. I think services based enterprise startups are probably different, but I don’t have much first hand experience with them – so this only for software. Here is my list of ten things an enterprise software needs to pay attention to early on:

1. An enterprise software startup needs a REALLY compelling value proposition for anyone in the enterprise to even consider them.
• That means you are addressing a KNOWN pain point that hasn’t been addressed before
• or are drastically lowering the cost of doing things already being done
2. Unless you provide direct value to the business user (in other words, value that they can see for themselves or their direct reports) you’ll be talking to IT.
• Even if you provide direct value to business user, unless you can deliver COMPLETELY over the web, you’ll still be talking to IT.
3. If you are talking to IT – you need for them to understand what is in it for them, not just for the business users.
4. Points 2 and 3 mean only people that directly benefit from your product will be willing to pay for it – a really important lesson.
5. Channel sales work only if you are replacing a known functionality. If you are doing something new, channels just don’t work. Later, when you have identified the customer, established the need and method of selling you can start using channels.
6. Direct selling to enterprises is expensive – really expensive. I believe that is the real reason enterprise software is so expensive – multiple proof of concepts, mutiple stakeholders that keep changing along with the requirements, integration with legacy systems and exitsing work processes – all make for a long, expensive process. If you aren’t selling a product that costs 6-7 figures, you just can’t  afford it.
7. The big vendors really have a lock on enterprise customers, and have really good, well-oiled sales\marketing machines. They may not have the best products, or even the right products – but even today no one gets fired for buying IBM – and IBM (or any of the other large enterprise software vendors) is really good at selling at all levels.
8. If you have a tactical product (solving a relatively low level business problem), you’ll have trouble getting people’s attention, if you are selling a strategic product the large vendors will generate so much static that it will be hard to get a message across.
9. The platform you build on matters – everything else being equal I would choose Linux, Java with a Browser UI (maybe MS if I had to, but only for the UI). This works in almost any enterprise (of course the exact version of each could be a problem too). On the sophisticated UI side, I worry a bit about Flash, but HTML5  still isn’t widely adopted – so if possible I would try and use neither. If push came to shove, I would probably opt for HTML5.
10. Analysts play an important role – if you are doing something that is already on their radar, make sure they know about you (and like you). If what you are doing isn’t on their radar – the best thing you can do is convince them that it is the next big thing.

Invention, Innovation and Knowledge Work

I heard a corporate presentation yesterday about innovation – I won’t mention the context, but suffice it to say it was the standard stuff that you hear from many corporations about the need to innovate or die – and then find out the group tasked with innovation isn’t even the size of a small startup – and reports somewhere deep in the bowels of the company.

But it did get me thinking abou the difference between invention and innovation. Putting it glibly -

Invention – From nothing to Aha!

Innovation – From Aha to Ka-Ching!

It used to be that you needed invention before you could do innovation. Get the basic science right – and then figure how to leverage that into products that people want. In social software, I think what we are seeing is that the innovation (actual products being used in the field) is the invention – though many of better social software systems heed well know truths about organization and social interactions. The reason this works is that the internet makes it cheap and easy to try out and iterate on new ways for people to collaborate – testing a hypothesis on real users. There is also enough funding available (through venture capital) to do this on a large scale. Back when I was at IBM research we used call this “research in the marketplace”.

This doesn’t hold in the corporate environment – which is why it is lagging and we don’t see knowledge worker productivity products with the blowout success of a Facebook or a Twitter. Not that I don’t think Facebook-like or Twitter-like product won’t be valuable in the enterprise – it is that they won’t make a significant difference to overall knowledge worker productivity.

The problem is that an eco-system isn’t there to easily try new way of increasing productivity for knowledge workers and there no real VC support. An organization will only adopt tools that have been proven to work. So how do we get around that “chicken and egg” problem? How can we increase “research in the marketplace” for the enterprise market to increase the productivity of knowledge workers?