Disruptive Enterprise Platform Sales: Why Buy Anything, Buy Mine, Buy Now - Part III

This is the third and the last post in the three-post series on challenges associated with sales of disruptive platforms such as Big Data and how you can effectively work with your prospects and others to mitigate them. If you missed the previous posts the first post was about “why buy anything” and the second post was about “why buy mine." This post is about “why buy now."

Platform sales is often times perceived as a solution looking for a problem a.k.a hammer looking for a nail. In most cases your prospects don’t have a real urgency to buy your platform making it difficult for you to make them commit on an opportunity. There are a few things that you could do to deal with this situation:

Specific business case

It’s typical for vendors to create a business case positioning their solutions to their prospects. These business cases include details such as solution summary, pricing, ROI etc. If you’re a platform vendor not only you have to create this basic business case but you will also have to go beyond that. It’s relatively hard to quantify ROI of a platform since it doesn’t solve a specific problem but it could solve many problems. It is extremely difficult to quantify the impact of lost opportunities. If your prospect doesn’t buy anything do they lose money on lost opportunities? Traditional NPV kind of analysis goes for a toss for such situations.

As a vendor not only you will have to articulate the problems (scenarios/use cases) that you identified leading up to this step but you might also have to include more scenarios that were not specifically discussed during the evaluation phase. Getting a validation from the business on expected return on their investment while fulfilling their vision is crucial since your numbers will most likely get challenged when your prospect creates its own business case to secure necessary investment to buy your platform.

Leveraging the excitement

What seemed like a problem when you worked with a variety of people inside your prospect’s organization may not seem like a problem in a few weeks or months. It’s very important in platform sales cycle not to lose momentum. Find a champion of your pilot keep socializing the potential of your platform inside your prospect’s organization as much as you can while you work on commercials of your opportunity. People should be talking about your disruptive platform and wanting to work with you. Cease that moment to close it.

Knowing who will sign the check

Platform sales are convoluted. People who helped you so far may not necessarily help you with the last step—not that they don’t want to but they may not be the buyers who will sign the check. It’s not uncommon in enterprise software sales to have influencers who are not the final buyers but the buyers do have somewhat defined procurement process for standard solutions. When it comes to buying a platform many buyers don’t quite understand why they should be spending money on disruptive platform that may or may not necessarily solve a specific problem.

To complicate this further, for disruptive technology, it typically tends to get cheaper as it becomes more mature. This gives your prospect one more reason to wait and not buy your platform now. As I mentioned in the previous post your focus should never be on pricing (unless of course you are the best and cheapest vendor by a wide margin) but on immediate returns, no lost opportunities, and helping your prospect gain competitive differentiation in their industry.

Despite of working with your prospect for a while helping them define problems and piloting your platform to prove out the value proposition, you might get asked again to do things all over again. There are two ways to mitigate this situation: a) involve buyers early on in the sales process and not at the very end so that they are part of the journey b) work aggressively with your influencers to establish appropriate communication channels with the buyers so that it’s the influencer’s voice they hear and not yours.

Happy selling!

Photo Courtesy: Wierts Sebastien  

Amazon's Cloud Price Reduction, A Desire To Compete Hard And Move Up The Value Chain

Recently Google slashed price for their cloud offering. Amazon, as expected, also announced their 42nd price reduction on their cloud offerings since its inception. Today, Microsoft also announced price reduction for their Azure offerings.

Unlike many other people I don't necessarily see the price reduction by Amazon as waging a price war against the competition.

Infrastructure as true commodity: IaaS is a very well understood category and Amazon, as a vendor, has strong desires to move up in the value chain. This can only happen if storage and computing become true commodity and customers value vendors based on what they can do on top of this commodity storage and computing. They become means to an end and not an end itself.

Amazon is introducing many PaaS like services on top of EC2. For example, RedShift is the fastest growing service on EC2. These services create stickiness for customers to come back and try out and perhaps buy other services. These services also create a bigger demand for the underlying cloud platform. Retaining existing customers and acquiring new customers with as little barrier as possible are key components of this strategy.

Reducing hardware cost: The hardware cost associated with computing and storage have gradually gone down. Speaking purely from financial perspective existing assets depreciate before they are taken out from service. Also, new hardware is going be cheaper than the old hardware (at the original cost). If you do pass on the cost advantage to your customers it should help you reduce price and compete at the same or a little less margin. However, hardware cost is a fraction of overall operations cost. In the short term, Amazon being a growth company will actually spend a lot more on CapEx and not just OpEx to invest and secure the future.

Economies of scale: The cost to serve two computing units is not the sum of cost to serve two one computing units. There are many economies of scales in play such as increasing data-center utilization, investment in automation, and better instance management software. Confidence in predicting minimum base volume and reducing fluctuations also gives Amazon better predictability to manage elasticity. As the overall volume goes up the elasticity or the fluctuations as percentage of overall volume go down. On top of that offerings such as Reserved Instances also are a good predictor of future demand. Amazon views Reserved Instances as how banks view CDs but many customers are looking for a "re-finance" feature for these Reserved Instances when price drops. These economic and pricing implications are great to watch.

To offer competitive pricing to win against  incumbents and make it almost impossible for new entrants to compete on the same terms is absolutely important but it would be foolish to assume it is the sole intent behind the price reduction.

Photo courtesy: Flickr

Challenges For On-premise Vendors Transitioning To SaaS

As more and more on-premise software vendors begin their journey to become SaaS vendors they are going to face some obvious challenges. Here's my view on what they might be.

The street is mean but you can educate investors

Sharp contrast between Amazon and Apple is quite clear. Even though Amazon has been in business for a long time with soaring revenue in mature categories the street sees it as a high growth company and tolerates near zero margin and surprises that Jeff Bezos brings in every quarter. Bezos has managed to convince the street that Amazon is still in heavy growth mode and hasn't yet arrived. On the other hand despite of Apple's significant revenue growth—in mature as well as in new disruptive categories—investors treat Apple very differently and have crazy revenue and margin expectations.

Similarly, traditional pure SaaS companies such as Salesforce is considered a high growth company where investors are focused on growth and not margins. But, if you're an on-premise vendor transitioning to SaaS the street won't tolerate a hit on your margins. The street would expect mature on-premise companies to deliver on continuous low double digit growth as well as margins without any blips and dips during their transition to SaaS. As on-premise vendors change their product, delivery, and revenue models investors will be hard on them and stock might take a nosedive if investors don't quite understand where the vendors are going with their transition. As much as investors love the annuity model of SaaS they don't like uncertainty and they will punish vendors for lack of their own understanding in the vendor's model. It's a vendor's job to educate investors and continuously communicate with them on their transition.

Isolating on-premise and SaaS businesses is not practical

Hybrid on-premise vendors should (and they do) report on-premise and subscription (SaaS) revenue separately to provide insights to investors into their revenue growth and revenue transition. They also report their data center related cost (to deliver software) as cost of revenue. But, there's no easy way, if at all there's one, to split and report separate SG&A costs for their on-premise and SaaS businesses. In fact combined sales and marketing units are the weapons incumbents on-premise vendors have to successfully transition to SaaS. More on that later in this post.

The basic idea behind achieving economies of scale and to keep the overall cost down (remember margins?) is to share and tightly integrate business functions wherever possible. Even though vendors sometime refer to their SaaS and on-premise businesses as separate lines of businesses (LoBs), in reality they are not. These LoBs are intertwined that report numbers as single P&L.

Not being able to charge more for SaaS is a myth

Many people I have spoken to assume that SaaS is a volume-only business and you can't charge customers what you would typically charge your customers in your traditional license and maintenance revenue business model. This is absolutely not true. If you look at some of the deal sizes and length of SaaS contracts of pure SaaS companies they do charge a premium when they have unique differentiation regardless of volume. Customers are not necessarily against paying premium - for them it is all about bringing down their overall TCO and increasing their ROI with reduced time to value. If a vendor's product and its delivery model allow customers to accomplish these goals they can charge them premium. In fact in most cases this could be the only way out. As a vendor transitioning from on-premise to SaaS their cost is going to go up; they will continue to invest into building new products and transitioning existing products and they will significantly assume the cost of running operations on behalf of their customers to deliver software as a service. They not only will have to grow their top-line to meet the growth expectations but to offset some of the cost to maintain the margins.

Prime advantage on-premise incumbents have over SaaS entrants

So, what does work in favor of on-premise vendors who are going through this transition?

It's the sales and marketing machine, my friends.

The dark truth about selling enterprise software is you need salespeople wearing suits driving around in their BMWs to sell software. There's no way out. If you look at high growth SaaS companies they spend most of what they earn on sales and marketing. Excluding Workday there is not much difference in R&D cost across vendors, on-premise or SaaS. Workday is building out its portfolio and I expect to see this cost go down in a few years.

Over a period of time, many on-premise vendors have built a great brand and achieved amazing market penetration. As these vendors go through SaaS transition they won't have to spend as much time and money educating the market and customers. In fact I would argue they should thank other SaaS vendors for doing the job for them. On-premise vendors have also built an amazing sales machine with deep relationship with customers and reliable sales processes. If they can maintain their SG&A numbers they will have enough room to deal with a possible initial hit on revenue and additional cost they would incur as they go through this transition.

Be in charge of your own destiny and be aggressive

It's going to be a tough transition regardless of your loyal customer base and differentiating products. It will test the execution excellence of on-premise vendors. They are walking on a tight rope and there's not much room to make mistakes. The street is very unforgiving.

Bezos and Benioff have consistently managed to convince the street they are high growth companies and should be treated as such. There's an important lesson here for on-premise vendors. There is no reason to label yourself an on-premise vendor simply making a transition. You could do a lot more than that; invest into new disruptive categories and rethink existing portfolio. Don't just chase SaaS for its subscription pricing but make an honest and explicit attempt to become a true SaaS vendor. The street will take a notice and you might catch a break.

Rise Of Big Data On Cloud

Growing up as an engineer and as a programmer I was reminded every step along the way that resources—computing as well as memory—are scarce. The programs were designed on these constraints. Then the cloud revolution happened and we told people not to worry about scarce computing. We saw rise of MapReduce, Hadoop, and countless other NoSQL technology. Software was the new hardware. We owe it to all the software development, especially computing frameworks, that allowed developers to leverage the cloud—computational elasticity—without having to understand the complexity underneath it. What has changed in the last two to three years is a) the underlying file systems and computational frameworks have matured b) adoption of Big Data is driving the demand for scale out and responsive I/Os in the cloud.

Three years back, I wrote a post, The Future Of The BI In Cloud where I had highlighted two challenges of using cloud as a natural platform for Big Data. The first one was to create a large scale data warehouse and the second was lack of scale out computing for I/O intensive applications.

A year back Amazon announced RedShift, a data warehouse service in the cloud, and last week they announced high I/O instances for EC2. We have come a long way and more and more I look at the current capabilities and trends, Big Data, at scale, on the cloud, seems much closer to reality.

From a batched data warehouse to interactive analytic applications:

Hadoop was never designed for I/O intensive applications, but Hadoop being a compelling computational scale out platform developers had a strong desire to use it for their data warehousing needs. This made Hive and HiveQL popular analytic frameworks but this was a sub optimal solution that worked well for batch loads and wasn't suitable for responsive and interactive analytic applications. Several vendors realized there's no real reason to stick to the original style of MapReduce. They still stuck to the HDFS but significantly invested into alternatives to Hive which are way faster.

There are series of such projects/products that are being developed on HDFS and MapReduce as a foundation but by adding special data management layers on top of it to run interactive queries much faster compared to plain vanilla Hive. Some of those examples are Impala from Cloudera and Apache Drill from MapR (both based on Dremel), HAWQ from EMC, Stinger from Hortonworks and many other start-ups. Not only vendors but the early adopters such as Facebook created Hive projects such as Presto, an accelerated Hive, which they recently open sourced.

From raw data access frameworks to higher level abstraction tools: 

As vendors continue to build more and more Hive alternatives I am also observing vendors investing in higher level abstraction frameworks. Pig was amongst those first higher level frameworks that made it easier to express data analysis programs. But, now, we are witnessing even higher layer rich frameworks such as Cascading and Cascalog not only to write SQL queries but write interactive programs in higher level languages such as Clojure and Java. I'm a big believer in empowering developers with right tools. Working directly against Hadoop has a significant learning curve and developers often end up spending time on plumbing and other things that can be abstracted out in a tool. For web development, popularity of Angular and Bootstrap are examples of how right frameworks and tools can make developers way more efficient not having to deal with raw HTML, CSS, and Javascript controls.

From solid state drives to in-memory data structures: 

Solid state drives were the first step in upstream innovation to make I/Os much faster but I am observing this trend go further where vendors are investing into building in-memory resident data management layers on top of HDFS. Shark and Spark are amongst the popular ones. Databricks has made big bets on Spark and recently raised $14M. Shark (and hence Spark) is designed to be compatible with Hive but designed to run queries 100x times faster by using in-memory data structures, columnar representation, and optimizing MapReduce not to write intermediate results back to disk. This looks a lot like MapReduce Online which was a research paper published a few years back. I do see a UC Berkeley connection here.

Photo courtesy: Trey Ratcliff

A Journey From SQL to NoSQL to NewSQL

Two years back I wrote that the primary challenge with NoSQL is that it's not SQL. SQL has played a huge rule in making relational databases popular for the last forty years or so. Whenever the developers wanted to design an(y) application they put an RDBMS underneath and used SQL from all possible layers. Over a period of time, the RDBMS grew in functions and features such as binary storage, faster access, clusters, sophisticated access control etc. and the applications reaped these benefits. The traditional RDBMS became a non-fit for cloud-scale applications that fundamentally required scale at whole different level. Traditional RDBMS could not support this scale and even if they could it became prohibitively expensive for the developers to use it. Traditional RDBMS also became too restrictive due to their strict upfront schema requirements that are not suitable for modern large scale consumer web and mobile applications. Due to these two primary reasons and a lot more other reasons we saw the rise of NoSQL. The cloud movement further fueled this growth and we started to see a variety of NoSQL offerings.

Each NoSQL store is unique in which how a programmer would access it. NoSQL did solve the scalability and flexibility problems of a traditional database, but introduced a set of new problems, primary ones being lack of ubiquitous access and consistency options, especially for OLTP workload, for schema-less data stores.

This has now led to the movement of NewSQL (a term initially coined by Mat Aslett in 2011) whose working definition is: "NewSQL is a class of modern relational database management systems that seek to provide the same scalable performance of NoSQL systems for OLTP workloads while still maintaining the ACID guarantees of a traditional single-node database system." NewSQL's focus appears to be on gaining performance and scalability for OLTP workload by supporting SQL as well as custom programming models and eliminating cumbersome error-prone management tasks such as manual sharding without breaking the bank. It's a good first step in the direction of a scalable distributed database that supports SQL. It doesn't say anything about mixed OLTP and OLAP workload which is one of the biggest challenges for the organizations who want to embrace Big Data.

From SQL to NoSQL to NewSQL, one thing that is common: SQL.

Let's not underestimate the power of a simple non-procedural language such as SQL. I believe the programmers should focus on what (non-procedural such as SQL) and not how. Exposing "how" invariably ends up making the system harder to learn and harder to use. Hadoop is a great example of this phenomenon. Even though Hadoop has seen widespread adoption it's still limited to silos in organizations. You won't find a large number of applications that are exclusively written for Hadoop. The developers first have to learn how to structure and organize data that makes sense for Hadoop and then write an extensive procedural logic to operate on that dataset. Hive is an effort to simplify a lot of these steps but it still hasn't gained desired populairty. The lesson here for the NewSQL vendors is: don't expose the internals to the applications developers. Let a few developers that are closer to the database deal with storing and configuring the data but provide easy ubiquitous access to the application developers. The enterprise software is all about SQL. Embracing, extending, and augmenting SQL is a smart thing to do. I expect all the vendors to converge somewhere. This is how RDBMS and SQL grew. The initial RDBMS were far from being perfect but SQL always worked and the RDBMS eventually got better.

Distributed databases is just one part of the bigger puzzle. Enterprise software is more about mixing OLAP and OLTP workload. This is the biggest challenge. SQL skills and tools are highly prevalent in this ecosystem and more importantly people have SQL mindset that is much harder to change. The challenge to vendors is to keep this abstraction intact and extend it without exposing the underlying architectural decisions to the end users.

The challenge that I threw out a couple of years back was:

"Design a data store that has ubiquitous interface for the application developers and is independent of consistency models, upfront data modeling (schema), and access algorithms. As a developer you start storing, accessing, and manipulating the information treating everything underneath as a service. As a data store provider you would gather upstream application and content metadata to configure, optimize, and localize your data store to provide ubiquitous experience to the developers. As an ecosystem partner you would plug-in your hot-swappable modules into the data stores that are designed to meet the specific data access and optimization needs of the applications."

We are not there, yet, but I do see  signs of convergence. As a Big Data enthusiast I love this energy. Curt Monash has started his year blogging about NewSQL. I have blogged about a couple of NewSQL vendors, NimbusDB (NuoDB) and GenieDB, in the past and I have also discussed the challenges with the OLAP workload in the cloud due to its I/O intensive nature. I am hoping that NewSQL will be inclusive of OLAP and keep SQL their first priority. The industry is finally on to something and some of these start-ups are set out to disrupt in a big way.

Photo Courtesy: Liz

Minimize Regrets And Not Failures

While I ponder on 2012 and plan for 2013, I always keep the regret minimization framework (watch the short video clip above) in back of my mind. Of course luck plays a huge part in people's success, but we owe it a lot to Jeff Bezos. We probably wouldn't have seen Amazon.com and we most certainly would not have seen EC2. No one predicted anything about Amazon being a key cloud player. A few years back Twitter didn't exist and Facebook was limited to college kids. I do make plans but I have stopped predicting since I will most certainly get it wrong.

"Plans are useless, but planning is indispensable." - Dwight Eisenhower

I use regret minimization framework not only as a long-term thinking tool but also to make decisions in short-term. It helps me assess, prioritize, and focus on right opportunities. While long-term thinking is a good thing, I strongly believe in setting short term goals, meeting them, and more importantly cherishing them. If you're not minimizing regret you're minimizing fear of failures. I don't fear failures, I celebrate them; they're a learning opportunity. As Bill Cosby put it, "In order to succeed, your desire for success should be greater than your fear of failure."

All the best with your introspection and indispensable planning for 2013. Focus on the journey, the planning, and not the destination, the plan.

Building And Expanding Enterprise Software Business In Brazil

While in Brazil, describing his country, one of my friends said, "We have all the natural resources that we need to be a self-sufficient country and we have had no natural disasters such as earthquakes and hurricanes. The only disaster that we had: we lost the worldcup."

This pretty much summarizes Brazil.

A helipad in front of my hotel in São Paulo.

On one side, São Paulo, the seventh largest city in the world, has one the largest per capita helipads in the world where the rich people don't like to drive around in traffic in cheap cars to avoid getting kidnapped at stop lights. On the other side, it is one hell of a city, just like Mumbai - large, organized chaos, and money. It is growing and it's growing fast. While income inequality has been on a steep rise in emerging economies as well as in the western world, it is declining in the Latin American countries, especially in Brazil.

If you're thinking of building or expanding enterprise software business in Brazil, now is the time. This is why:

Developing to a developed economy

Brazil has been boxed into BRIC economies but in reality it behaves more like a developed economy with lingering effects of a developing economy. Even though corruption is rampant in Brazil, it exists at much higher level and a common man typically doesn't suffer as miserably as he/she would suffer in other countries such as India. Being a resourceful country, there are all kinds of jobs. The bureaucracy will break and the infrastructure will also catch up very soon due to the soccer world cup in 2014 followed by the Olympics in 2016. Don't apply your BRIC strategy to Brazil. Consider Brazil as a developed nation and aggressively expand.

Courtesy: Economist

Stronger middleclass

Middleclass has money and they are willing to spend. Brazilian tax laws are the most complex laws that I have ever seen.  Even though the global retail brands are present in Brazil, they are outrageously expensive. Making a weekend trip to Miami to shop is quite common. Even after paying for a plane ticket and hotels it is cheaper shop in the US. The retailers in Brazil are trying to better understand this behavior and the global brands are also looking at several different ways to market to this middleclass. As an ISV this is a gold mine that you should not be ignoring.

Local to Global

Following the nation's growth many local companies in Brazil are aspiring to go global, establishing their business in developed economies. Local ISVs neither have scale nor features to support these efforts. These companies (typically mid to large) are looking at global ISVs for help, and yes, they are willing to spend.

Then you ask, if it is this obvious, why aren't global ISVs already doing this. They are. It's obvious, but it is not that easy. These are the challenges you would run into:

Complex localization

Many global ISVs have given up localizing their software for the Brazilian market. The tax laws are extremely complex and so are other processes. If you are truly interested in the Brazilian market you need to build from scratch in Brazil for Brazil. Hire local talent, empower them, and educate them on your global perspective. Linux and related open source software talent is plentiful in São Paulo. These developers are also excited about the cloud are are building some amazing stuff. I would also suggest to either hire or partner with local domain experts as consultants, who can work with a product manager, to truly understand the nuts and bolts of local processes, laws, and regulations.

Rough sales cycles

Selling into large accounts is not easy. Work with partners for a joint go-to-market solution or have them lead or participate in your sales cycle. The sales cycle is not fair and square and the purchase decisions are not just based on merits of your offering. Even if customer likes a product, commercial discussion are a huge drag, from the sponsor, to buyer, to all the way up to purchasing. Be patient and take help of local experts to navigate these roads.

My taxi driver watching a live soccer game while driving

Language and culture

Speaking Portuguese is pretty much a requirement to get anything done. But, if you speak Spanish, you could get around and also pick up a little bit of conversational Portuguese. English-only approach won't work. Do not even attempt. Also, Brazilians don't like to be called Latin Americans. They like to be called Brazilians; avoid any Latin American references. While you are there, learn a thing or two from an average Brazilian about fitness. Unlike Americans, the Brazilians are not into junk food. At a churrascaria, they eat salad followed by meat followed by some more meat. If you wonder why they are so fit, especially in Rio, this diet perhaps explains. They do enjoy their lives and sip Cachaça at the beach, but they are damn serious about working out.

With Yammer, Microsoft Begins Its Journey From Collaborative To Social

Confirming what we already knew, today Microsoft announced they are acquiring Yammer for $1.2 billion in cold cash. Here's a blog post by David Sacks, the CEO of Yammer.

Microsoft doesn't report a revenue breakdown for their individual products but SharePoint is believed to be one of the fastest growing products with annual revenue of more than $1 billion. Regardless of how Microsoft markets and positions SharePoint, it has always been collaboration software and not really social software. Microsoft does seem to understand the challenges it faces in moving their portfolio of products to the cloud, including SharePoint. Microsoft also understands value of having end users on their side even though SharePoint is sold as enterprise software. Microsoft's challenges in transitioning to the cloud are similar to the ones faced by other on-premise enterprise software vendors.

But, I really admire Microsoft's commitment by not giving up on any of these things. Skype's acquisition was about reaching those millions of end users and they continue to do that with their acquisition of Yammer. Going from collaborative to social requires being able to play at the grassroots level in an organization as opposed to a top down push and more importantly being able to create and leverage network effects. It's incredibly difficult to lead in with an on-premise solution retrofitted for cloud to create network effects. Native cloud solutions do have this advantage. Yammer will do this really well while helping Microsoft to strengthen SharePoint as a product and maintain its revenue without compromising margins. If Microsoft executes this well, they might unlock a solution for their Innovator's Dilemma.

With Yammer, Microsoft does have an opportunity to fill in the missing half of social enterprise by transforming productivity silos into collaborative content curation. As a social enterprise software enthusiast, I would love to see it happen, sooner rather than later.

Instagram: $1B. Yammer: $1.2B.And that, my friends, is why you do enterprise software.
— Aaron Levie (@levie) June 15, 2012

At personal level, I am excited to see the push for social in enterprise software and a strong will and desire to cater to the end users and not just the decision makers.  I hope that more entrepreneurs recognize that enterprise software could be social, cool, and lucrative. This also strengthens market position for the vendors such as Box and Asana.

It's impressive what an incumbent can do when they decide to execute on their strategy. Microsoft is fighting multiple battles. They do have the right cards. It's to be seen how they play the game.

Data Is More Important Than Algorithms

Netflix Similarity Map

In 2006 Netflix offered to pay a million dollar, popularly known as the Netflix Prize, to whoever could help Netflix improve their recommendation system by at least 10%. A year later Korbel team won the Progress Prize by improving Netflix's recommendation system by 8.43%. They also gave the source code to Netflix of their 107 algorithms and 2000 hours of work. Netflix looked at these algorithms and decided to implement two main algorithms out of it to improve their recommendation system. Netflix did face some challenges but they managed to deploy these algorithms into their production system.

Two years later Netflix awarded the grand prize of $1 million to the work that involved hundreds of predictive models and algorithms. They evaluated these new methods and decided not to implement them. This is what they had to say:

"We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment. Also, our focus on improving Netflix personalization had shifted to the next level by then."

This appears to be strange on the surface but when you examine the details it totally makes sense.

The cost to implement algorithms to achieve incremental improvement isn't simply justifiable. While the researchers worked hard on innovating the algorithms Netflix's business as well as their customers' behavior changed. Netflix saw more and more devices being used by their users to stream movies as opposed to get a DVD in mail. The main intent behind the million dollar prize for Netflix was to perfect their recommendation system for their DVD subscription plan since those subscribers carefully picked the DVDs recommended to them as it would take some time to receive those titles in mail. Customers wanted to make sure that they don't end up with lousy movies. Netflix didn't get any feedback regarding those titles until after their customers had viewed them and decided to share their ratings.

This customer behavior changed drastically when customers started following recommendations in realtime for their streaming subscription. They could instantaneously try out the recommended movies and if they didn't like them they tried something else. The barrier to get to the next movie that the customers might like significantly went down. Netflix also started to receive feedback in realtime while customers watched the movies. This was a big shift in user behavior and hence in recommendation system as customers moved from DVD to streaming.

What does this mean to the companies venturing into Big Data?

Algorithms are certainly important but they only provide incremental value on your existing business model. They are very difficult to innovate and way more expensive to implement. Netflix had a million dollar prize to attract the best talent, your organization probably doesn't. Your organization is also less likely to open up your private data into the public domain to discover new algorithms. I do encourage to be absolutely data-driven and do everything that you can to have data as your corporate strategy including hiring a data a scientist. But, most importantly, you should focus on your changing business — disruption and rapidly changing customer behavior — and data and not on algorithms. One of the promises of Big Data is to leave no data source behind. Your data is your business and your business is your data. Don't lose sight of it. Invest in technology and more importantly in people who have skills to stay on top of changing business models and unearth insights from data to strengthen and grow business. Algorithms are cool but the data is much cooler.

4 Big Data Myths - Part II

This is the second and the last part of this two-post series blog post on Big Data myths. If you haven't read the first part, check it out here.

Myth # 2: Big Data is an old wine in new bottle

I hear people say, "Oh, that Big Data, we used to call it BI." One of the main challenges with legacy BI has been that you pretty much have to know what you're looking for based on a limited set of data sources that are available to you. The so called "intelligence" is people going around gathering, cleaning, staging, and analyzing data to create pre-canned "reports and dashboards" to answer a few very specific narrow questions. By the time the question is answered its value has been diluted. These restrictions manifested from the fact that the computational power was still scarce and the industry lacked sophisticated frameworks and algorithms to actually make sense out of data. Traditional BI introduced redundancies at many levels such as staging, cubes etc. This in turn reduced the the actual data size available to analyze. On top of that there were no self-service tools to do anything meaningful with this data. IT has always been a gatekeeper and they were always resource-constrained. A lot of you can relate to this. If you asked the IT to analyze traditional clickstream data you became a laughing stroke.

What is different about Big Data is not only that there's no real need to throw away any kind of data, but the "enterprise data", which always got a VIP treatment in the old BI world while everyone else waited, has lost that elite status. In the world of Big Data, you don't know which data is valuable and which data is not until you actually look at it and do something about it. Every few years the industry reaches some sort of an inflection point. In this case, the inflection point is the combination of cheap computing — cloud as well as on-premise appliances — and emergence of several open computing data-centric software frameworks that can leverage this cheap computing.

Traditional BI is a symptom of all the hardware restrictions and legacy architecture unable to use relatively newer data frameworks such as Hadoop and plenty of others in the current landscape. Unfortunately, retrofitting existing technology stack may not be that easy if an organization truly wants to reap the benefits of Big Data. In many cases, buying some disruptive technology is nothing more than a line item in many CIOs' wish-list. I would urge them to think differently. This is not BI 2.0. This is not a BI at all as you have known it.


Myth # 1: Data scientist is a glorified data analyst

The role of a data scientist has exponentially grown in its popularity. Recently, DJ Patil, a data scientist in-residence at Greylock, was featured on Generation Flux by Fast Company. He is the kind of a guy you want on your team. I know of a quite a few companies that are unable to hire good data scientists despite of their willingness to offer above-market compensation. This is also a controversial role where people argue that a data scientist is just a glorified data analyst. This is not true. Data scientist is the human side of Big Data and it's real.

If you closely examine the skill set of people in the traditional BI ecosystem you'll recognize that they fall into two main categories: database experts and reporting experts. Either people specialize in complicated ETL processes, database schemas, vendor-specific data warehousing tools, SQL etc. or people specialize in reporting tools, working with the "business" and delivering dashboards, reports etc. This is a broad generalization, but you get the point. There are two challenges with this set-up: a) the people are hired based on vendor-specific skills such as database, reporting tools etc. b) they have a shallow mandate of getting things done with the restrictions that typically lead to silos and lack of a bigger picture.

The role of a data scientist is not to replace any existing BI people but to complement them. You could expect the data scientists to have the following skills:

• Deep understanding of data and data sources to explore and discover the patterns at which data is being generated. 
• Theoretical as well practical (tool) level understanding of advanced statistical algorithms and machine learning.
• Strategically connected with the business at all the levels to understand broader as well deeper business challenges and being able to translate them into designing experiments with data.  
• Design and instrument the environment and applications to generate and gather new data and establish an enterprise-wide data strategy since one of the promises of Big Data is to leave no data behind and not to have any silos.

I have seen some enterprises that have a few people with some of these skills but they are scattered around the company and typically lack high level visibility and an executive buy-in.

Whether data scientists should be domain experts or not is still being debated. I would strongly argue that the primary skill to look for while hiring a data scientist should be how they deal with data with great curiosity and asking a lot of whys and not what kind of data they are dealing with. In my opinion if you ask a domain expert to be a data expert, preconceived biases and assumptions — knowledge curse —  would hinder the discovery. Being naive and curious about a specific domain actually works better since they have no pre-conceived biases and they are open to look for insights in unusual places. Also, when they look at data in different domains it actually helps them to connect the dots and apply the insights gained in one domain to solve problems in a different domain.

No company would ever confess that their decisions are not based on hard facts derived from extensive data analysis and discovery. But, as I have often seen, most companies don't even know that many of their decisions could prove to be completely wrong had they have access to right data and insights. It's scary, but that's the truth. You don't know what you don't know. BI never had one human face that we all could point to. Now, in the new world of Big Data, we can. And it's called a data scientist.

Photo courtesy: Flickr

4 Big Data Myths - Part I

It was cloud then and it's Big Data now. Every time there's a new disruptive category it creates a lot of confusion. These categories are not well-defined. They just catch on. What hurts the most is the myths. This is the first part of my two-part series to debunk Big Data myths.

Myth # 4: Big Data is about big data

It's a clear misnomer. "Big Data" is a name that sticks but it's not just about big data. Defining a category just based on size of data appears to be quite primitive and rather silly. And, you could argue all day about what size of data qualifies as "big." But, the name sticks, and that counts. The insights could come from a very small dataset or a very large data set. Big Data is finally a promise not to discriminate any data, small or large.

It has been prohibitively expensive and almost technologically impossible to analyze large volumes of data. Not any more. Today, technology — commodity hardware and sophisticated software to leverage this hardware — changes the way people think about small and large data. It's a data continuum. Big Data is not just about technology, either. Technology is just an enabler. It has always been. If you think Big Data is about adopting new shiny technology, that's very limiting. Big Data is an amalgamation of a few trends - data growth of a magnitude or two, external data more valuable than internal data, and shift in computing business models. The companies mainly looked at their operational data, invested into expensive BI solutions, and treated those systems as gold. Very few in a company got very little value out of those systems.

Big Data is about redefining what data actually means to you. Examine the sources that you never cared to look at before, instrument your systems to generate the kind of data that are valuable to you and not to your software vendor. This is not about technology. This is about completely new way of doing business where data finally gets the driver's seat. The conversations about organizations' brands and their competitors' brands are happening in social media that they neither control nor have a good grasp of. At Uber, Bradly Voytek, a neuroscientist is looking at interesting ways to analyze real-time data to improve the way Uber does business. Recently, Target came under fire for using data to predict future needs of a shopper. Opportunities are in abundance.

Myth # 3: Big Data is for expert users    

The last mile of Big Data is the tools. As technology evolves the tools that allow people to interact with data have significantly improved, as well. Without these tools the data is worth nothing. The tools have evolved in all categories ranging from simple presentation charting frameworks to complex tools used for deep analysis. With rising popularity and adoption of HTML 5 and people's desire to consume data on tablets, the investment in presentation side of the tools have gone up. Popular javascript frameworks such as D3 have allowed people to do interesting things such as creating a personal annual report. Availability of a various datasets published by several public sector agencies in the US have also spurred some creative analysis by data geeks such as this interactive report that tracks money as people move to different parts of the country.

The other exciting trend has been the self-service reporting in the cloud and better abstraction tools on top of complex frameworks such as Hadoop. Without self-service tools most people will likely be cut off from the data chain even if they have access to data they want to analyze. I cannot overemphasize how important the tools are in the Big Data value chain. They make it an inclusive system where more people can participate in data discovery, exploration, and analysis. Unusual insights rarely come from experts; they invariably come from people who were always fascinated by data but analyzing data was never part of their day-to-day job. Big Data is about enabling these people to participate - all information accessible to all people.

Coming soon in the Part II: Myth # 2 and Myth # 1.

Learning From Elevators To Design Dynamic Systems

Elevators suck. They are not smart enough to know which floor you might want to go. They aren't designed to avoid crowding in single elevator. And they make people press buttons twice, once to call an elevator and then to let it know which floor you want to go to. This all changed during my recent trip to Brazil when I saw the newer kind of elevators.

These elevators have a common button panel outside in the lobby area of a high rise building. All people are required to enter their respective floor numbers and the machine will display a specific elevator number that they should get into. Once you enter into an elevator you don't press any numbers. In fact the elevators have no buttons at all. The elevator would highlight the floor numbers that it would stop at. That's it! I love this redesigned experience of elevators. It solves a numbers of problems. The old style elevators could not predict the demand. Now the system exactly knows how many people are waiting at what floors wanting to go where. This allows the system to optimize the elevator experience based on several variables and criteria such as speed, priority, even distribution, power conservation etc. This also means an opportunity to write interesting algorithms for these elevators.

This is how I want ALL the systems to be - smart, adaptive, and dynamic. Just like this elevator I would like to see the systems, especially the cloud and the analytics, to anticipate the needs of the end users as opposed to following their commands. The context is the key to the success of delivering what users would expect. If the systems are designed to inquire about the context — directly or indirectly, just like asking people to push buttons before they get into an elevator — they would perform more intelligently. Some location-based systems have started to explore this idea, but it's just the beginning. This also has significant impact on designing collaborative recommendation systems that could help the end users find the right signal in the ever increasing noise of social media.

The very idea of the cloud started with the mission to help users with elasticity of the commodity resources without having users to learn a different interface by giving them a unified abstraction. If you had two elevators in a lobby, you wouldn't use this. But, for a high rise with a few elevators, the opportunities are in abundance to optimize the system to use the available resources to provide the best experience to the people, the end users.

Self-configuring and self-healing dynamic systems have been a fantasy, but as the cloud becomes more mature, dynamic capabilities to anticipate the needs of an application and its users are not far fetched. Computing and storage are commodity on the cloud. I see them as resources just like elevators. Instead of people pushing buttons at the eleventh hour I would prefer the cloud take a driver's seat and becomes much smarter at anticipating and managing applications, platforms, and mixed workload. I want the cloud to take this experience to the next level by helping developers develop such adaptive and dynamic applications. I almost see it as a scale issue, at system as well as at human level. If the cloud does promise scale I expect it to go beyond the commodity computing. This is why PaaS excites me more than anything else. That's a real deal to make a difference.

Simple Workflow Service - Amazon Adding One Enterprise Brick At Time

Yesterday, Amazon announced a new orchestration service called Simple Workflow Service. I would encourage you to read the announcement on Werner's blog where he explains the need, rationale, and architecture. The people I spoke to had mixed reactions. One set of people described this as a great idea and were excited that the developers can now focus on writing domain-specific code as opposed to writing plumbing code to orchestrate their actual code. The other set of people felt that this service creates a new cloud lock-in making it difficult for the developers to switch from one cloud to another as well as being able to interoperate at the orchestration level.

I believe this is a brilliant idea for a variety of reasons. Orchestration has always been painful. Ask the developers who have been involved in managing task execution across a cluster that required them to code for load balancing, handling exceptions, restarting hung processes, tracking progress etc. This is not a core competency the most developers have but they do end up writing such code due to lack of better alternative. The frameworks such as WS-BPEL were not designed to run in cloud-like environments and there has been no single standard REST orchestration framework out there that people could use.

From a vendor's perspective, I admire Amazon's ability to keep innovating via such services that differentiate them as a leading cloud vendor. As computing becomes more and more commodity, competing based on price alone isn't a good idea. If you're a cloud vendor you need to go above and beyond the traditional IaaS attributes even though you excel in all of them. I also see PaaS eventually bleeding into IaaS as IaaS continues to become a commodity. As far as PaaS goes, federated or otherwise, we're barely scratching the surface.

I don't see this service as a cloud lock-in but it certainly makes EC2 more attractive and sticky. I would be concerned if Amazon were to force the developers to use their SWS for orchestration. This is their version of how they think orchestration should be done and the developers can opt in if they want. And kudos to them to think beyond their cloud. The folks who worry about cloud lock-ins also talk about Amazon not following the standards. I believe that we should not create standards for the sake of creating standards. I am a believer in first showing that something works and later, if there's enough interest, figure out a way to standardize it. All these talks about standard-first even before you write that first line of code doesn't make any sense.

It's yet to be seen how this service turns out, but this is a huge step forward for getting more enterprise software customers onboard. Orchestration is one of the most chronic problems of enterprise software and with the challenges of a hybrid landscape to be able to orchestrate across on-premise and cloud-based solutions, this service is certainly a step in the right direction. Right Scale has been using a Ruby workflow Ruote for their workflow needs and now they orchestrate these workflows using SWS  to achieve fault tolerance and concurrency. As you can see, Amazon has opened up a gold mine for start-ups. The back-end execution has always been challenging. Now, there is an opportunity to write your own enterprise grade workflow engine or scheduler that runs in the cloud.

Wrong Side Of The IT Ecosystem

I find it ridiculous that people are blaming Apple for job creation in China as opposed to in the US. People are also debating how US might in-source some of these manufacturing jobs to compete with China who has sophisticated manufacturing abilities and large skilled labor force supporting these operations. They are all missing the point. This is a wrong debate.

The US lost manufacturing jobs to other countries a long time ago. I find it amusing that people expect the high-tech companies such as Apple to create manufacturing jobs in the US. If Apple were to even consider this option we would not have seen the tremendous success of Apple as a company and its products. What Apple created is an ecosystem of people and companies that are doing amazing things with their platform and their devices. It's a different kind of ecosystem and America should focus on that innovation as opposed to bringing those manufacturing jobs back.

On one side we are whining about the loss of manufacturing jobs and on the other side we have shortage of skilled IT workforce. Try hiring a good developer in the Silicon Valley and you'll understand what I mean. And yet as a nation we are behind in retraining our existing workforce, attracting students to engineering majors, and fixing our immigration policy for highly skilled foreign workers to meet the increased demand of IT-driven jobs. And, of course, while we wait, Apple is quadrupling its IT investment in India.

America should not play the manufacturing game with China or for that matter with anyone else. We are at such a loss. Let's play the game that we know we can win — technology-driven innovation. When I work with customers' on daily basis I come across so many opportunities that we are not looking at. We can use the technology, that we have built, to our advantage in the industries such as healthcare, agriculture, public sector etc. A combination of cloud and mobility could take us long way.

We're looking at the wrong side of the IT ecosystem. I don't expect the hi-tech companies to hire low-tech workers in the US. But I do expect hi-tech companies to create jobs in the US at the other end of the ecosystem via the opportunities to consume their technology and innovate in a different sector. A lot of people are missing this point. I'm talking about an ecosystem where Apple has paid out more than $4 billion to the developers. Why are we not talking about these jobs? Apple has more than $100 billion in cash, but what doesn't get much discussed is that a large part of this cash is overseas. Given the current US tax laws, Apple can't/won't bring this cash back into the US. This might make Apple acquiring or investing overseas. We do have an opportunity to reform the tax laws to deal with such a global situation (that we never encountered before) to encourage the hi-tech companies to invest into R&D in the US and not overseas.

When you look at the big picture, having a job is merely one piece of contributing to good standards of living. What about access to affordable healthcare and college education? There's a significant opportunity to apply technology built in America to innovate in these areas. We are barely scratching the surface of what's possible in healthcare as well as in education. We are living in such an antiquated and inefficient system.

Another industry that has seen less than desired technology innovation is agriculture. Take a trip down to central California to see the potential. At 2008 Olympics in China, Michael Phelps winning 8 gold medals was obviously the biggest highlight for us, but people in Watsonville were super excited because China allowed the US to export Watsonville strawberries for the Olymipcs. Recently, India relaxed the laws (that are still being challenged) to allow 100% foreign investment in the retail sector opening up the doors for Wallmarts of the world. Any guess what's the biggest challenge in retail operations in India? A non-existent cold supply chain and lack of reliable infrastructure. We take a lot of things for granted — nationwide freeways, strong governing bodies such as FDA, and size of the country. We do have an opportunity to excel in certain agriculture areas and employ a lot of Americans. We need to recognize what our real strength is and look forward as opposed to look backwards.

I am a geek and a technology enthusiast, and perhaps a little naive. But, I know for sure, we aren't pushing the technology envelope as much as we should.

Photo: Looking Towards Tomorrow

Loving What I Do For Living

A few months back, I was helping a very large customer of ours to help simplify as well as automate their process of trading financial instruments. During one of my many visits to their office, I met a person who was trying to explain to me his job in supporting the people that are involved in this super complex process. I always ask a lot of questions — until they're totally annoyed and ready to kick me out of the room — to get a complete understanding of the business rationale behind whatever they're thriving for and their personal motivation behind it. Something unusual happened at this meeting. Instead of getting into the gory technical details of how they get things done, he chose to tell me a short and simple story.

"You know, um.. there's this early morning meeting everyday that Peter goes to with a bunch of other people. They all gather around a large table in a dimly lit conference room with a bunch of printed spreadsheets, a laptop, and a large calculator. Peter has a cup of coffee in one hand and a cigarette in the other hand talking to people who have coffee cups in their one hand and cigarettes in the other hand. This is their lives. I am concerned about Peter and I want him to stop smoking. Can you please help me?"

Now, this is the job that I love that makes me get out the bed and run for it. This is the human side of enterprise software. It's not boring.

Photo Courtesy: Jane Rahman

Early Signs Of Big Data Going Mainstream

Today, Cloudera announced a new $40m funding round to scale their sales and marketing efforts and a partnership with NetApp where NetApp will resell Cloudera's Hadoop as part of their solution portfolio. These both announcements are critical to where the cloud and Big Data are headed.

Big Data going mainstream: Hadoop and MapReduce are not only meant for Google, Yahoo, and fancy Silicon Valley start-ups. People have recognized that there's a wider market for Hadoop for consumer as well as enterprise software applications. As I have argued before Hadoop and Cloud is a match made in heaven. I blogged about Cloudera and the rising demand of data-centric massive parallel processing almost 2.5 years back, Obviously, we have come a long way. The latest Hadoop conference is completely sold out. It's good to see the early signs of Hadoop going mainstream. I am expecting to see similar success for companies such as Datastax (previously Riptano) which is a "Cloudera for Cassandra."

Storage is a mega-growth category: We are barely scratching the surface when it comes to the growth in the storage category. Big data combined with the cloud growth is going to drive storage demand through the roof and the established storage vendors are in the best shape to take advantage of this opportunity. I wrote a cloud research report and predictions this year with a luminary analyst Ray Wang where I mentioned that cloud storage will be a hot cake and NoSQL will skyrocket. It's true this year and it's even more true next year.

Making PaaS even more exciting: PaaS is the future and Hadoop and Cassandra are not easy to deploy and program. Availability of such frameworks at lower layers makes PaaS even more exciting. I don't expect the PaaS developers to solve these problems. I expect them to work on providing a layer that exposes the underlying functionality in a declarative as well as a programmatic way to let application developers pick their choice of PaaS platform and build killer applications.

Push to the private cloud: Like it or not, availability of Hadoop from an "enterprise" vendor is going to help the private cloud vendors. NetApp has a fairly large customer base and their products are omnipresent in large private data centers. I know many companies that are interested in exploring Hadoop for a variety of their needs but are somewhat hesitant to go out to a public cloud since it requires them to move their large volume of on-premise data to the cloud. They're more likely to use a solution that comes to their data as opposed to moving their data to where a solution resides.

Bangalore Embodies The Silicon Valley

I spent a few days in Bangalore this month. This place amazes me every single time I visit it. Many people ask me whether I think Bangalore has potential to be the next Silicon Valley. I believe, it's a wrong question. There's some seriously awesome talent in India, especially in Bangalore. Don't copy the Silicon Valley. There are so many intangibles that Bangalore won't get it right. And there's no need to copy. Create a new Silicon Valley that is the best of both worlds.

If you want some good reading on what makes silicon valley the Silicon Valley, read the essay "How to be Silicon Valley" by Paul Graham. Bangalore does have some of these elements - diversity, clusters, a large number of expats etc. It's quickly becoming a true cosmopolitan city in India. You don't need to know the local language (Kannada) to live there. It does have a few good colleges such as IIM and IISC, but no IIT. The real  estate boom in Bangalore is a clear indicator of what's going on in the city with regards to the spending power of the middle class and the upper middle class. Most large IT multinationals have a campus in Bangalore. The companies such as Accenture have more people in Bangalore than in the US.

So, what's wrong?

Lack of entrepreneurial mentorship

If you go back to the roots of the early success of the Silicon Valley you will find that the venture capitalists community mentored the entrepreneurs to bring innovation to life. Steve Jobs had an idea, but no business plan. Some of the entrepreneurs became serial entrepreneurs and some became the investors who in turn mentored other entrepreneurs. This cycle continued. I don't see this in Bangalore. Not only the VC funding is not easily accessible (more on this below), but there are no early investors that I see are spotting the trends and mentoring the entrepreneurs.

I spoke to many entrepreneurs in Bangalore. Let me tell you - they do not lack the entrepreneurial spirit. They are hungry and they are foolish. And they are chomping at the bit to work on an exciting idea, but they do lack someone to mentor them and take them through the journey.

Where have all the designers gone?

A couple of years ago I was invited at the National Institute of the Design (NID), a premier design school in India, for a guest lecture. They told me that design is not a discipline that easily attracts good talent in India. They are competing with the engineering schools. India lacks designers. This is the age of experience start-ups. A very few engineers have the right design mindset. If they want to be successful, they absolutely need to work with the designers who are impossible to find and hire. This talent gap is hurting to manifest the vision of a founder into a product that the consumers would love to use. Flipkart and Red Bus are my favorite start-ups but they are few and far between.

Math and Science would only take you so far

It's not just Math and Science that has created the Silicon Valley. It's the right balance of creativity, business acumen, and engineering talent. The schools in India, even today, are not set up to let students be more creative. They are still fixated on Math and Science since they guarantee good jobs. The Silicon Valley entrepreneurs followed their dreams. In the US, it's about studying what you like and chase a career that you are happy with and not to pick a certain kind of education just because they provide good jobs. Unfortunately, creativity is hard to teach. It's ingrained into the culture, society, and the systems. If India has to get this right, this needs to start at the education and a support system that has a place for jobs other than Math and Science.

I have been following the education reforms in India and private sector investment into K-12 schools. They are encouraging. I don't believe Bangalore or India for that matter will have math or science issue anytime soon, but it will certainly have entrepreneurial issues to jump start new companies and manage their ever growing engineering workforce. I was invited to speak at IIM Ahmedabad, one of the best business schools in India. During my conversation with the faculties, I was told that the most pressing issue for the elite business schools in India is to scale their efforts to create the new class of mid-management that can manage the rapidly growing skilled workforce.

Obama keeps saying more and more people in the US should study math and science to be competitive. I don't believe that's the real competition. The real competition is what can you do if you did know math and science or if you had access to people who knew it.

Lack of streamlined access to capital

A lot has been written about this obvious issue and I don't want to beat this further. I just want to highlight that despite of all the money that the individuals and large corporations have earned in India, a very little is being invested into venture capital since the VC framework, processes, and the regulations aren't as streamlined. It's not a level playing field. In the Silicon Valley, the venture money is commodity. If you have a great idea, team, or a product, the investors will run after you to invest into your company. Bangalore is far from this situation. But it shouldn't have to be. What's missing is not the available money but a class of people who can run local funds by investing into the right start-ups. Most US VC firms have set up shops in India, but I don't think that's enough to foster innovation at the grassroots level. Bangalore needs Indian firms to recognize the need for a local VC community that can work with the system to make those funds available to the entrepreneurs.

The picture: I took this picture inside one of the SAP buildings in Bangalore during the week before Diwali.

Disrupt Yourself Before Others Disrupt You: DVD To Streaming Transition Is Same As On-Premise To Cloud

Recently, Netflix separated their streaming and DVD subscription plans. As per Netflix's forecast, they will lose about 1 million subscribers by the end of this quarter. The customers did not like what Netflix did. A few days back, Netflix's CEO, Reed Hastings, wrote a blog post explaining why Netflix separated their plans. He also announced their new brand, Qwikster, which will be a separate DVD service from Netflix's streaming website. These two services won't share the queues and movie recommendations even if you subscribe to both of them. A lot has been said and discussed about how poorly Netlflix communicated the overall situation and made wrong decisions.

I have no insider information about these decisions. They might seem wrong in short term but I am on Netflix's side and agree with the co-founder Marc Randolph that Netflix didn't screw up. I believe it was the right thing to do, but they could have executed it a little better. Not only I am on their side, but I see parallels between Netflix's transition from DVD to steaming and on-premise enterprise ISVs' transition from on-premise to cloud. The on-premise ISVs don't want to cannibalize their existing on-premise business to move to the cloud even if they know that's the future, but they don't want to wait long enough to be in a situation where they run out of money and become irrelevant before the transition.

So, what can these on-premise ISV's learn from Netflix's decisions and mistakes?

Run it as a separate business unit, compete in the right category, and manage street's expectations:

Most companies run their business as single P&L and that's how the street sees it and expects certain revenue and margins. Single P&L muddies the water.The companies have no way of knowing how much money they are spending on a specific business and how much revenue it brings in. In many cases, there is not even an internal separation between different business units. Setting up a separate business unit is a first step to get the accounting practices right including tracking cost and giving the right guidance to the street. DVD business is like maintenance revenue and the streaming is like license revenue. The investors want to know two things: you're still a growth company (streaming) and you still have enough cash coming in (DVD business) to tap into the potential to grow.

Netflix faces competition in streaming as well as in their DVD business, but the nature of competition is quite different. For the enterprise ISVs competing with on-premise vendors is quite different than competing with SaaS vendors. The nature of business — cost structure, revenue streams, ecosystem, platform, anti-trust issues, marketing campaigns, sales strategy — is so different that you almost need a separate organization.

Prepare yourself to acquire and be acquired:

Netflix could potentially acquire a vendor in the streaming business or in the DVD business and that makes it easy for them to integrate. This is even more true in the case of ISVs since most of the on-premise ISVs will grow into the cloud through acquisitions. If you're running your SaaS business as a separate entity, it is much easier to integrate the new business from technology as well as business perspective.

Just as you could acquire companies, you should prepare yourself for an exit as well. Netflix could potentially sell the DVD unit to someone else. This will be a difficult transaction if their streaming business is intertwined with their DVD business. The same is true for the enterprise ISVs. One day, they might decide to sell their existing on-premise business. Running it as a separate business entity makes it much easier to attract a buyer and sell it as a clean transaction.

Take your customers through the journey: 

This is where Netflix failed. They did not communicate to the customers early on and ended up designing a service that doesn't leverage existing participation of the customers such as recommendations and queues. There is no logical reason why they cannot have a contract in place between two business units to exchange data, even if these two units are essentially separate business entities. The ISVs should not make this mistake. When you move to the cloud, make sure that your customers can connect to their on-premise systems. Not only that, you need to take care of their current contracts and extend them to the cloud if possible and make it easy for them to transition. Don't make it painful for your customers. The whole should be great than the sum of its parts.

Run your business as a global brand:

Learn from P&G and GE. They are companies made up of companies. They do run these sub-companies independently with a function to manage them across. It does work. Netflix has a great brand and they will retain that. As an on-premise ISV you should consider running your on-premise and cloud businesses as sub-brands under single brand umbrella. Branding is the opposite of financials; brand is a perception and financials is a reality. Customers care for the brand and service and the street cares for the financials. They seem to be very closely related to each other for a company looking inside-in but from an outside-in perspective they are quite different. There is indeed a way to please them both. This is where the most companies make wrong decisions.

Freemium Is The New Piracy In The SaaS World

It is estimated that approximately 41% of revenue, close to $53 billion, is "lost" in software piracy. This number is totally misleading since it assumes that all the people who knowingly or unknowingly pirated software would have bought the software at the published price had they not pirated it. RIAA also applies the same nonsense logic to blow the music piracy number way out of proportion. The most people who pirate software are similar to the people who pirate music. They may not necessarily buy software at all. If they can't pirate your software, they will pirate something else. If they can't do that, they will find some other alternative to get the job done.

Fortunately, some software companies understand this very well and they have a two-pronged approach to deal with this situation: prevent large scale piracy and leverage piracy when you can't prevent it. If an individual has access to free (pirated) software, as a vendor, you're essentially encouraging an organic ecosystem. The person who pirated your software is more likely to make a recommendation to continue using it when he/she is employed by a company that cannot and will not pirate. This model has worked extremely well. What has not been working so well and what the most on-premise vendors struggle with is the unintentional license usage or revenue leakage. Customers buy on-premise software through channels and deploy to large number of users. Most on-premise software are not instrumented to prevent unintentional license usage. The license activation, monitoring, and compliance systems are antiquated in most cases and cannot deal with this problem. This is very different than piracy because the most corporations, at least in the western world, that deploy the on-premise software want to be honest but they have no easy way to figure out how many licenses have beed used.

In the SaaS world, this problem goes away. The cloud becomes the platform to ensure that the subscriptions are paid for and monitored for continuous compliance. You could argue that there is no license leakage since there are no licenses to deal with. But, what about piracy? Well, there's no piracy either. This is a bad thing. Even though a try before buy exists, there's no organic grass-roots adoption of your software (as a service) since people can't pirate. In many countries where software piracy is rampant, the internet access is not ubiquitous and bandwidth is still limited. This creates one more hurdle for the people to use your software.

So, what does this mean to you?

SaaS ISV: It is very important for you to have a freemium model that is country-specific and not just a vanilla try-before-buy. You need to get users start using your service for free early on and make it difficult for them to move away when they work for someone who can pay you. Even though you're a SaaS company, consider a free on-premise version that provides significant value. Evernote is a great example of this strategy. It shouldn't surprise you that people still download software, pirated or otherwise. Don't try to change their behavior, instead make your business model fit to their needs. As these users become more connected and the economics work in their favor, they will buy your service. It's also important to understand that the countries where piracy is rampant, people are extremely value conscious.

On-premise ISV: Don't lose your sleep over piracy. It's not an easy problem to solve but do make sure that you're doing all you can to prevent it. Consider a freemium business model where you're providing a clean and free version to your users. If the users can get enough basic value from a free version, they are less likely to pirate a paid version. What you absolutely must do is to fix your license management systems to prevent unintentional license usage. Help yourself by helping your customers who want to be honest. The cloud is a great platform to collect, clean, and match all the license usage data. You have a little or no control over customers' landscapes but you do have control over your own system in the cloud as long as there's a little instrumentation embedded in your on-premise software and a hybrid architecture that connects your on-premise software to the cloud. In nutshell you should be able to manage your licenses the way SaaS companies manage their subscriptions. There are plenty of other benefits of this approach including the most important benefit being a SaaS repository of your customers and their landscapes. This would help you better integrate your future SaaS offerings and acquisitions as well as third-part tools that you might use to run your business.