Numbersense: How to Use Big Data to Your Advantage
Kaiser Fung
McGraw-Hill (2013)
How to cope with an information blizzard that has become a data tsunami
I agree with an observation by Mark Twain: “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies and statistics.’” There has been an abundance of books and articles published in recent years that discuss the emergence and significance of Big Data. According to Kaiser Fung, “Big Data is real, and its impact will be massive. At the very least, we are all consumers of data analyses. We must learn to be smarter consumers. What we need is Numbersense< .” In his opinion, those who generate the best analyses of data possess both technical ability and business acumen…and “Numbersense< is the third dimension.” What exactly is it? “Numbersense< is that noise in your head when you see bad data or bad analysis. It’s the desire and the persistence to get close to the truth. It’s the wisdom of knowing when to make a U-turn, when to press on, but mostly when to stop. It’s the awareness of where you came from, and where you’re going. It’s gathering clues, and recognizing decoys. The talented ones can find their way from A to Z with fewer wrong turns. Others struggle and get lost in the maze, possibly never finding Z. The best way to nurture Numbersense is by direct practice or by learning from others. I wrote this book to help you get started. Each chapter is inspired by a recent news item in which someone made a claim and backed it up with data.” When I first read this explanation, I really did not fully “get it” but after reading the book, I appreciated as well as understood what Numbersense< is…and isn’t. In a phrase, I view it as “street smarts for data consumers.”
These are among the dozens of passages that caught my attention and were highlighted with my Sharpie ACCENT pen, shared to provide a sense of the thrust and flavor of Kaiser Fung’s lively and eloquent narrative:
o On Basic Requirements: “Numbersense< is that bit of skepticism, urge to probe, and desire to verify. It's having the truffle hog's nose to hunt the delicacies. Developing Numbersense< takes training and patience. It is essential to know a few basic statistical concepts. Understanding the nature of means, medians, and percentile ranks is important. Breaking down ratios into components facilitates clear thinking. Ratios can also be interpreted as weighted averages, with those weights arranged by rules of inclusion and exclusion. Missing data must be carefully vetted, especially when they are substituted with statistical estimates. Blatant fraud, while difficult to detect, is often exposed by inconsistency." (Page 53) o On the Problem with the Problem: “Shrouded in the fog of war, we are losing sight of the problem we are trying to solve. [For example,] Obesity is not the adversary; rather, it is early death caused by obesity-related diabetes such as diabetes and stroke. This distinction is crucial. We can win the battle against obesity and still lose the battle on mortality.” (65)
o On Restaurants That Will Benefit from Coupon Promotions: “New entrants, with few loyal customers, have little to lose, and offer the best value for the money. Some established restaurants use the promotion to fill empty seats during the low season. These are the places that serve special menus offering anything but their regular fare. These are the places that can make money from those one-time meals. They aren’t counting on the return business.” (94)
o On Priming Effects (i.e. exposure to a stimulus influences a response to a later stimulus): “So many things could predispose one’s behavior. Multiple priming effects may be in effect simultaneously. The effect may only last for some unknown amount of time. Even after the effect has been demonstrated, people would not believe that they have been affected. The results from various experiments threaten the search for stable, logical, causal structures that explain our decisions. [Check out Daniel Kahneman’s Thinking, Fast and Slow.] The absence of explanations consigns statisticians to modeling correlations, an activity that is inherently prone to errors not curable by data infusion.” (125)
o On an Uninformed General Public: “Journalists on the economics beat have yet to wake up to Big Data. The Bureau of Labor Statistics makes public indices covering geographic regions, expenditure groups, and various definitions of inflation rates, and yet we seldom hear about them in the news. Disaggregation unwinds the averaging process, and the component indices tend to make more sense to us. When data id plentiful, we should appreciate the diversity of its components. Two strategies that sometimes backfire are averaging and filtering. The former stamps out the variety while the latter casts dark shadows.” 171)
o Final Thoughts: “I can’t leave you with the idea that everyone must become data analysts to survive the era of Big Data. That is not the logical conclusion of this book. I do warn you that the wide availability of data will bring confusion and invite mischief. I hope you won’t take data at face value ever again, and you see the power of looking under the hood.” (201)
I agree with Fung that the need to develop Numbersense< is more urgent now than ever before as what was once characterized as an information blizzard has since become a data tsunami. Here in a single volume is about all the information, insights, and counsel anyone needs to do that. As Kaiser Fung correctly suggests, in the new world of Big Data, “there is no escape from people hustling numbers.” True, but knowing how to make better decisions using better information is not only possible...it is imperative. Years ago, after parents of Harvard students vehemently protested against a tuition increase, then president Derek Bok responded, “If you think education is expensive, try ignorance.”