Big Data: The All-Important 90/10 Rule

Big Data

Here is a brief excerpt from an article by Bernard Marr, featured by LinkedIn Pulse. To read the complete article, check out others, and sign up to receive email alerts, please click here.

* * *

In this post I outline my 90/10 rule for big data initiatives in businesses. The postwas first published in my column for Data Science Central.

The phenomenon of Big Data is giving us ever-growing volume and variety of data we which we can now store and analyze. Any regular reader of my posts knows that I personally prefer to focus on Smart Data, rather than Big Data – because the term places too much importance on the size of the data. The real potential for revolutionary change comes from the ability to manipulate, analyze and interpret new data types in ever-more sophisticated ways.

The SMART Data Framework

I’ve written previously about my SMART Data framework which outlines a step-by-step approach to delivering data-driven insights and improved business performance.

1. Start with strategy: Formulate a plan – based on the needs of your business

2. Measure metrics and data: Collect and store the information you need

3. Apply analytics: Interrogate the data for insights and build models to test theories

4. Report results: Present the findings of your analysis in a way that the people who will put them into effect will understand

5. Transform your business: Understand your customers better, optimize business processes, improve staff wellbeing or increase revenues and profits.

My work involves helping businesses use data to drive business value. Because of this I get to see a lot of half-finished data projects, mothballed when it was decided that external help was needed.

The biggest mistake by far is putting insufficient thought – or neglecting to put any thought – into a structured strategic approach to big data projects. Instead of starting with strategy, too many companies start with the data. They start frantically measuring and recording everything they can in the belief that big data is all about size. Then they get lost in the colossal mishmash of everything they’ve collected, with little idea of how to go about mining the all-important insights.

This is why I have come up with the 90/10 rule – When working with data, 90% of your time should be spent on a structured strategic approach, while 10% of your time should be spent “exploring” the data.

The 90/10 Rule

The 90% structured time should be used putting the steps outlined in the SMART Data framework into operation. Making a logical progression through an ordered set of steps with a defined beginning (a problem you need to solve), middle (a process) and an ending (answers or results).

This is after all why we call it Data Science. Business data projects are very much like scientific experiments, where we run simulations testing the validity of theories and hypothesis, to produce quantifiable results.

The other 10% of your time can be spent freely playing with your data – mining for patterns and insights which, while they may be valuable in other ways, are not an integral part of your SMART Data strategy.

Yes, you can be really lucky and your data exploration can deliver valuable insights – and who knows what you might find, or what inspiration may come to you? But it should always play second-fiddle to following the structure of your data project in a methodical and comprehensive way.

* * *

Here is a direct link to the complete article.

Bernard Marr is a Best-Selling Author, Keynote Speaker and Leading Business and Data Expert. He helps companies improve decision-making and performance using data. His latest book is Big Data: Using Smart Big Data, Analytics and Metrics To Make Better Decisions and Improve Performance.

Posted in

Leave a Comment





This site uses Akismet to reduce spam. Learn how your comment data is processed.