Big data in the age of the telegraph

Big Data (1854)
Here is a brief excerpt from an article written by Caitlin Rosenthal for The McKinsey Quarterly, published by McKinsey & Company. In it, she includes Daniel McCallum’s 1854 organizational design for the New York and Erie Railroad, a design that resembles a tree rather than a pyramid. It empowered frontline managers by clarifying data flows. Its special significance is that it is now believed to be the first modern organization chart.

To read the complete article, please click here.

* * *

In 1854, Daniel McCallum took charge of the operations of the New York and Erie Railroad. With nearly 500 miles of track, it was one of the world’s longest systems, but not one of the most efficient. In fact, McCallum found that far from rendering operations more efficient, the scale of the railroad exponentially increased its complexity.1

The problem was not a lack of information: the growing use of the telegraph gave the company an unprecedented supply of nearly real-time data, including reports of accidents and train delays.2 Rather, the difficulty was putting that data to use, and it led McCallum to develop one of the era’s great low-tech management innovations: the organization chart. This article presents that long-lost chart (see sidebar, “Tracking a missing org chart”) and shows how aligning data with operations and strategy—the quintessential modern management challenge—is a problem that spans the ages.
‘Big data,’ then and now

Just as information now floods into companies by the tera-, peta-, and exabyte, during the mid-19th century, governments, businesses, and universities produced and grappled with what one historian has called an “avalanche of numbers.”3 To be sure, McCallum’s rail lines may not have generated even a megabyte of information. But this was indeed big data for him and his senior deputies, who were managing a system of unprecedented proportions. Although the telegraph’s speed made more information available, organizing and acting on it became increasingly difficult. One delayed train, for example, could disrupt the progress of many others. And the stakes were high: with engines pulling cars in both directions along a single set of rails, schedule changes risked the deadly crashes that plagued 19th-century railroads.

As McCallum reflected, “A superintendent of a road 50 miles [long] . . . may be almost constantly upon the line engaged in the direction of its details.” But on railroads like his, which stretched for hundreds of miles, no individual manager could be responsible for all of the necessary schedule changes. (See exhibit, “The first modern organization chart,” for McCallum’s solution.)

Daniel McCallum created the first organization chart in response to the information problem hobbling one of the longest railroads in the world. In surprising contrast to today’s top-down organization pyramids, in McCallum’s chart the hierarchy was reversed: authority over day-to-day scheduling and operations went to the divisional superintendents down the line, who oversaw the five branch lines of the railroad. The reasoning: they possessed the best operating data, were closer to the action, and thus were best placed to manage the line’s persistent inefficiencies.

Each superintendent was responsible for the physical geography of the tracks and stations and for the men who moved along the rails: conductors, brakemen, and laborers. Coordinating activities between these two branches, the superintendents managed both the fixed depots and the rolling stock that moved between them.

Notes:

1 This article’s details on the railway’s operations and organizational thought come from Homer Ramsdell and D. C. McCallum, Reports of the President and Superintendent of the New York and Erie Railroad to the Stockholders, for the Year Ending September 30, 1855, New York, NY: Press of the New York and Erie Railroad Company, 1856.

2 Tom Standage quotes contemporaries who called the telegraph the “highway of thought” in The Victorian Internet: The Remarkable Story of the Telegraph and the Nineteenth Century’s On-line Pioneers, first edition, London, UK: Weidenfeld & Nicolson, 1998. An excellent recent account of the telegraph’s impact is Richard John, Network Nation: Inventing American Telecommunications, first edition, Cambridge, MA: Harvard University Press, 2010.

3 The phrase “avalanche of numbers” comes from Ian Hacking, writing on the spread of probabilistic and statistical reasoning, in The Taming of Chance, first edition, Cambridge, UK: Cambridge University Press, 1990.

* * *

To read the complete article, please click here.

Caitlin Rosenthal, an alumnus of McKinsey’s Houston office, is the Harvard–Newcomen postdoctoral fellow at the Harvard Business School. She wishes to acknowledge Michael Chui for his contribution to this article.

Posted in

Leave a Comment





This site uses Akismet to reduce spam. Learn how your comment data is processed.