Here is a brief article by Geoffrey Moore for LinkedIn. To read entertaining as well as informative articles by other thought leaders, please click here.

Photo: alengo / Getty Images

* * *

Big Data and Analytics are still a hot ticket, both in the venture community and in the IT budgets of selected verticals—including any of the e-commerce enterprises competing directly with Amazon, any of the online transaction companies voluntarily exposing their sites to increasingly clever purveyors of fraud, or the media properties seeking to move beyond print and broadcast to get a real foothold on the Web. What underlies all these pockets of deep investment? The search for clues among our digital footprints. It’s a whole new era for enterprise IT.

Traditional enterprise IT vendors with strong Business Intelligence practices have sought to position themselves as the natural bridge to this new era. But there is a lot of water to cross over here. Traditional BI sits on top of a data warehouse that aggregates a history of structured data from an enterprise’s own transaction systems. Big data analytics sits on top of a Hadoop data engine that aggregates log files from a partially overlapping set of transaction engines and seeks to extract inferences from them that can be acted upon in real time. Even more basically, BI is for people and is about the past, present, and future state of things, of nouns. Analytics is for machine learning and is about drawing inferences from the traces left by actions, by verbs. They really are as different as chalk and cheese.

The amazing thing about log files is that they really do let you see what people are doing. This is incredibly revealing if for no other reason than we have yet to develop masking behaviors on line to disguise what we are up to. Did you go to the site? Did you stay? Where did you linger? Did your mouse pause above a button? Did you leave quickly without participating? Did you buy anything? If so, what? And who else bought that? What other sites do we know they visited? What other ones did you visit? Who is in your cohort? Who has the most influence in your cohort? How many cohorts are you in?

Whether it is Omniture tracking the traffic on your web site or LinkedIn or Facebook drawing your social graph, there is much to learn about you just from the records in their logs of your comings and goings. A lot of people want to make this a privacy issue—which it certainly can become—but that is a red herring. Privacy can be protected, albeit through sacrifices on many sides. Perhaps more to the point, neither you nor I are really all that interesting, so our privacy, important though it may be to us personally, is not actually what this is all about. Instead it is about behavioral targeting, fraud prevention, business process reengineering, marketing optimization, health diagnostics, and the like. And at its best, as I said, it is amazing stuff.

So do not be fooled. This is not people drawing inferences from reports and charts. This is computational engines grinding through billions of data points in real time to extract probabilistic inferences that can move the yield on programmatic initiatives by a few percentage points. It is hardly personalization yet. But it does have big economic impact, enough to catapult the leaders in this field to the top of the stack, and leave the uninitiated grasping at straws.

That’s what I think. What do you think?

* * *

Geoffrey Moore is an author, speaker, and business advisor to many of the leading companies in the high-tech sector, including Cisco, Cognizant, Compuware, HP, Microsoft, SAP, and Yahoo! He divides his time between consulting on strategy and transformation challenges with senior executives and speaking internationally on those same topics. His latest book Escape Velocity: Free Your Company’s Future from the Pull of the Past, keeps this intent in mind and is the result of his years of experience working with large enterprises. He cordially invites you to check out the resources at his website by clicking here.

Link to my first interview of him

Link to my second interview of him