Unsupervised Machine Learning, Most Promising Ingredient Of Big Data

Orange (France Telecom), one of the largest mobile operators in the world, issued a challenge "Data for Development" by releasing a dataset of their subscribers in Ivory Coast. The dataset contained 2.5 billion records, calls and text messages exchanged between 5 million anonymous users in Ivory Coast, Africa. Various researchers got access to this dataset and submitted their proposals on how this data can be used for development purposes in Ivory Coast. It would be an understatement to say these proposals and projects were mind-blowing. I have never seen so many different ways of looking at the same data to accomplish so many different things. Here's a book [very large pdf] that contains all the proposals. My personal favorite is AllAborad where IBM researchers used the cell-phone data to redraw optimal bus routes. The researchers have used several algorithms including supervised and unsupervised machine learning to analyze the dataset resulting in a variety of scenarios.

In my conversations and work with the CIOs and LOB executives the breakthrough scenarios always come from a problem that they didn't even know existed or could be solved. For example, the point-of-sale data that you use for your out-of-stock analysis could give you new hyper segments using clustering algorithms such as k-means that you didn't even know existed and also could help you build a recommendation system using collaborative filtering. The data that you use to manage your fleet could help you identify outliers or unproductive routes using SOM (self organizing maps) with dimensionality reduction. Smart meter data that you use for billing could help you identify outliers and prevent thefts using a variety of ART (Adoptive Resonance Theory) algorithms. I see endless scenarios based on a variety of unsupervised machine learning algorithms similar to using cell phone data to redraw optimal bus routes.

Supervised and semi-supervised machine learning algorithms are also equally useful and I see them complement unsupervised machine learning in many cases. For example, in retail, you could start with a k-means to unearth new shopping behavior and end up with Bayesian regression followed by exponential smoothing to predict future behavior based on targeted campaigns to further monetize this newly discovered shopping behavior. However, unsupervised machine learning algorithms are by far the best that I have seen—to unearth breakthrough scenarios—due to its very nature of not requiring you to know a lot of details upfront regarding the data (labels) to be analyzed. In most cases you don't even know what questions you could ask.

Traditionally, BI has been built on pillars of highly structured data that has well-understood semantics. This legacy has made most enterprise people operate on a narrow mindset, which is: I know the exact problem that I want to solve and I know the exact question that I want to ask, and, Big Data is going to make all this possible and even faster. This is the biggest challenge that I see in embracing and realizing the full potential of Big Data. With Big Data there's an opportunity to ask a question that you never thought or imagined you could ask. Unsupervised machine learning is the most promising ingredient of Big Data.

To RIM: Don't Change The Strategy, Change The Rules

A lot has been said and discussed about RIM's downfall: indecisive leadership, inability to innovate at fast pace, and no clear path to recovery. I don't disagree at all with the analysis and the interpretation of the situation, but I do disagree with the conclusion that many people are drawing and vehemently disagree with their advice to RIM to keep trying to regain the smartphone market share. That train has left the station and RIM doesn't have a chance to catch up, even if they do everything that they could.

But RIM may have stumbled upon something that they probably least expected. It's the BlackBerry Messaging, popularly known as BBM. We got to see the power of BBM during the London riots. During my recent trip to India, I firsthand witnessed how much of people's lives depend on BBM. These people were sad, upset, and depressed due to a RIM infrastructure outage. This is a phenomenal success. The recipe behind this success is quite simple: provide free messaging that looks likes SMS that supports groups in a network. RIM has significantly leveraged network effects; BBM got better as more and more people used it. The sale of BlackBerry in India has gone viral. The consumers buy Blackberries since their friends have it so that they can chat with them for free and perhaps do their emails. These consumers don't use any apps at all! Their needs are quite simple. These phones are also priced well - the median price is somewhat around $200 for an unlocked phone. The Indian middle class and upper middle class have no issues shelling out this money to buy a BlackBerry. I talked to quite a few people and they are moving away from Android and iPhone to BlackBerry. Yes, that's right. If RIM can manage to introduce lower end versions of BlackBerry this will further fuel the growth.

May be, just may be, there's a category between smart and non-smart phones. For a large number of people in emerging economies making a phone call and staying in touch with their friends and family via text messages and email, and not paying too much for doing that are the driving reasons to purchase a right kind of a phone.

Let's briefly look at the history of RIM. It was the device of choice for email and calendering and perhaps still is for a lot of people. RIM myopically focused on going after the enterprise customers while iPhone and Android pulled the rug underneath them. RIM initially ignored and later underestimated the disruptive nature of this innovation. What started out as a consumer market, iPhone and Android easily crossed the chasm and entered into an enterprise and started replacing BlackBerry. We all know this story. But, something happened during this era of RIM: they ended up building a massively scalable and reliable enterprise class messaging infrastructure. This is an amazing feat of technical excellence. Building BlackBerry Messaging was a logical extension of leveraging this infrastructure. What if RIM uses this as a strength and not worry about competing in the smart OS area.

It's time to pivot.

Build a robust phone that is primarily driven off by BlackBerry Messaging and double down in emerging economies. Change the rules of the game and beat Nokia at its own strategy. Even better, spin off BlackBerry into two separate businesses: one that exclusively focuses on this strength and the other that embraces innovation by OEMing either Android or Windows or both and defend the handset as well as the services market share. I don't believe BlackBerry is cut out to innovate on a new smart phone OS quick enough to beat iOS or Android or an emerging contender, Windows phone. That would mean playing by your competitors' rules. If you learn one thing from Apple, it would be not to do this.

I don't need to tell you how many cellphones the Indians own and how many of those can buy a BlackBerry. This may not be an intended move, but this social effects driven business in emerging economies such as India as well as in other developed countries could be the second act for BlackBerry. Can other vendors replicate this? May be. Not many companies in the world can do what BlackBerry does with emails and messaging in general. Group messaging on a mobile device is a killer app in itself to drive the sales of handsets. Also, this works across the carriers and the geographies, essentially allowing RIM not to be threatened by a provider. SMS GupShup in India has been an extremely popular group messaging service. It's a validation that there is significant untapped potential for RIM.

Photo courtesy: NoHoDamon

Bottom Of The Pyramid – Nokia’s Second Act

The two-third of world’s 4.6 billon mobile users live in the emerging markets. Millions of these users live below the poverty line and are part of the bottom of the pyramid (BOP). Nokia is the market leader in these emerging markets, at least for now, with 34% market share. It’s clear from rapidly declining Nokia’s marketshare and an appointment of new CEO, Stephen Elop, that Nokia needs a second act. I believe the BOP is what could be the next big thing for Nokia.

The recent NYTimes story highlights a Nokia’s service, to supply commodity data to the farmers in India, using a text message. So far, 6.3 million people have signed up for this service. Nokia is planning to roll out this service, Life tools, in Nigeria as well. This is part of their Ovi mobile business.

I have written before on impact of cloud computing and mobile on the bottom of the pyramid and the importance of public policy innovation in emerging markets. The BOP is one of the biggest opportunities that Nokia currently has. Nokia has been losing marketshare in the smartphone category, and it is going to get increasingly difficult for Nokia to compete with Apple, Google, RIM, and now Microsoft. However, the very same vendors will find it equally difficult to move down the chain to compete with Nokia in the emerging markets.

One of the biggest business challenges to cater to the BOP is not a desire to market or a product to offer, but it is the lack of direct access to these consumers. The people at the BOP are incredibly difficult to reach. I have seen many go-to-market plans fail because it is either impossible or prohibitively expensive to market to these consumers. One of the biggest assets Nokia has is the relationship, the channel, with the people at the BOP. Now is the time to focus and leverage that channel by providing them with the content and the services that could be served on these phones via a strong platform, built for the BOP, and a vibrant ecosystem built around it.

My two cents: exit from the Smartphone category and double down the investment to serve the people at the bottom of the pyramid.

Nokia, that could be your second act.

Telcos Could Be The Future Enterprise Software Vendors For Small Businesses

Having worked on enterprise software product and go-to-market strategy for SMB (small and medium businesses), I can tell you that these are the most difficult customers to reach to, especially the S in SMB. It’s an asymmetric non-homogeneous market for which the cost of sales could go out of control if you don’t leverage the right channels. The competitive landscape varies from region to region and industry to industry. In many cases instead of competing against a company you would be competing against a human being with paper-based processes.

Tomorrow I am speaking at the Razorsight annual conference on the topic of cloud computing. I am excited to meet their customers, the telcos. While I prepare for my keynote, I can’t stop thinking about the challenges that the telcos face and the opportunities that they are not pursuing. My keynote presentation is about how telcos can leverage the cloud, but this blog post is about how telcos can become successful enterprise software vendors and market their solutions to small businesses.

There are very few things that are common across small businesses. They own a landline (at least for now) and they have Internet access, in many cases from the same vendor. I believe that the landlines will be more and more difficult to sell to these customers, but losing a channel – a relationship – would be even worse. If leveraged well, these relationships could be worth a lot more compared to the landline business as it stands today. Just think about it. Selling to small businesses is all about leveraging existing relationships with them. This channel is priceless.

What will it take for the telcos to market products to small businesses?

ISV acquisitions or VAR agreements: If telcos are bundling software, on-premise or SaaS, the telcos, as organizations, don’t necessarily have the skills or resources to make software for small businesses. This would mean a series of small and niche ISV acquisitions across geographical areas and industries and VAR agreements with current ISVs.

What kind of software can telcos bundle?

There are two kinds: horizontal and vertical. The examples of horizontal software are accounting, payroll, point of sale etc. Ask Intuit and they will tell you all about the horizontal cash cow. The vertical software is industry specific for the business that you are in. One of my favorite companies in this area is OpenTable. If you have made an online reservation at a restaurant you have most likely used their software. They had a successful IPO last year and they are on track to become a $100 million company.

Telcos should be doing all these things. They have cash and they can borrow cheap money to buy companies. Telcos also have an option to leverage the cloud, their own cloud in many cases, to provide SaaS solutions to small businesses. They can leap frog the on-premise ISVs who don’t have access to these customers and are sensitive to margin cannibalization.