Deep Learning
John D. Kelleher
MIT Press (September 2019)

A brilliant analysis of learning “on the other side of complexity”

This is one of the most valuable volumes in the MIT Press Essential Knowledge Series. According to Bruce Tidor (Professor of Biological Engineering and Computer Science at MIT), each volume offers “accessible, concise, beautifully produced pocket-size books on topics of current interest. Written by leading thinkers, the books in this series deliver expert overviews of the subjects that range from the cultural and the historical to the scientific and the technical.”

As you probably know already, deep learning is the subfield of machine learning that designs and evaluates training algorithms and architectures for modern neural network models. A deep neural network is a network that has multiple layers of hidden units (or neurons). In this volume, Kelleher provides an accessible introduction to the artificial intelligence technology that enables computer vision, speech recognition, machine translation, and driverless cars.

These are among the passages of greatest interest and value to me, also listed to indicate the nature and scope of Kelleher’s coverage:

o Artificial intelligence (Pages 6-8
o Why is Machine Learning Difficult? (16-22)
o The Key Ingredients of Machine Learning (22-25)
o Why Is Deep Learning So Successful? (30-34)
o Learning Model Parameters from Data (49-54)

o How an Artificial Neuron Processes Information (70-77)
o Why Is an Activation Function Necessary? (77-79 and 150-153)
o A Brief History of Deep Learning (101-133)
o Connectionism and Local versus Distributed Representations (129-141)
o Network Architectures: Convolutional and Recurrent Neural Networks (133-145)

o Layer-Wise Pretraining Using Autoencoders (145-148)
o The Virtuous Cycle: Better Algorithms, Faster Hardware, BiggercData (153-155)
o Convolutional Neural Networks (160-163)
o Rec ufrrfent Neural Networks (170-177)
o Gradient Descent (187-198)

o Backpropagation: The Two-Stage Algorithm (210-215)
o Backpropagation: Backpropagating the [delta]s (216-222)
o Big Data Driving Algorithmic Innovations (232-237)
o The Emergence of New Models (237-240)
o The Challenge of Interpretability (245-248)

These are am0ng John Kelleher’s concluding thoughts: “It is likely, that for the last few years, you have unknowingly been using deep learning models on a daily basis. A deep learning model is probably being invoked every time you use an internet search engine, a machine translation system, a face recognition system on your camera or social media website, or use a speech interface to a smart device. What is potentially more worrying is that the trail of data and metadata that you leave as you move through the online world is also being processed and analyzed using deep learning models. That is why it is so important to understand what deep learning is, how it works, what it is capable of, and its current limitations.”

The value of the information, insights, and counsel that he provides is incalculable. Amazon now sells a paperbound edition of Deep Learning for only $13.99. That’s not a bargain; that’s a steal. I think that this is a must-read for all senior-level executives as well as for those now preparing for a career in business or have only recently embarked on one.

* * *

Here are two suggestions while you are reading Deep Learning: First, highlight key passages. Also,  perhaps in a notebook kept near-at-hand (e.g. Apica Premium C.D. Notebook A5), record your comments, questions, and action steps (preferably with deadlines). Pay special attention to key insights (text in white on black pages) that are strategically inserted throughout the lively and eloquent narrative.

These two simple tactics — highlighting and documenting — will expedite frequent reviews of key material later.