What is Deep Learning?

There has been excessive hype and a mountain of misinformation about Deep Learning (DL), which could lead some to believe it’s just more nonsense to ignore. It isn’t and it shouldn’t be.

I’m not a computer scientist, let alone an AI researcher, but these five books have helped me get my head around what DL and AI are and what they aren’t, as well as where they may be headed:

– Artificial Intelligence (Russell and Norvig)

– Neural Network Design (Hagan)

– Deep Learning and Neural Networks (Heaton)

– Deep Learning (Goodfellow et al.)

– Deep Learning with R (Chollet and Allaire)

The last of these is a very unusual book that cleverly combines a ton of computer code with psychology and good old common sense, and does so in clear, lively prose. It may be a sort of computer science equivalent of a page turner. I won’t comment in depth about Deep Learning or the book – I am not qualified to do so – but instead will conclude by citing some notable passages from Deep Learning with R:

“A concise definition of the field [AI] would be as follows: the effort to automate intellectual tasks normally performed by humans. As such, AI is a general field that encompasses machine learning and deep learning, but that also includes many more approaches that don’t involve any learning.”

“Machine learning is tightly related to mathematical statistics, but it differs from statistics in several important ways. Unlike statistics, machine learning tends to deal with large, complex data sets…for which classical statistics such as Bayesian analysis would be impractical. As a result, machine learning, and especially deep learning, exhibits comparatively little mathematical theory – maybe too little – and is engineering oriented. It’s a hands-on discipline in which ideas are proven empirically more often than theoretically.”

“Deep learning is a specific subfield of machine learning: a new take on learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations. The deep in deep learning…stands for this idea of successive layers of representations.”

“Nowadays the name neural network exists purely for historical reasons – it’s an extremely misleading name because they’re neither neural nor networks. In particular, neural networks have hardly anything to do with the brain.”

“…deep learning is more an art than a science. We can provide guidelines that suggest what is likely to work or not work on a given problem, but, ultimately every problem is unique; you’ll have to evaluate different strategies empirically. There is currently no theory that will tell you in advance precisely what you should do to optimally solve a problem. You must iterate.”

“Trying to use machine learning to beat markets, when you only have access to publicly available data, is a difficult endeavor, and you’re likely to waste your time and resources with nothing to show for it…Machine learning…is applicable to datasets where the past is a good predictor of the future.”

“…artificial intelligence isn’t about replacing our own intelligence with something else, it’s about bringing into our lives and work more intelligence – intelligence of a different kind. In many fields, but especially in creative ones, AI will be used by humans as a tool to augment their own capabilities: more augmented intelligence than artificial intelligence.”

“Understanding the problem domain so as to be able to determine what to attempt to predict, given what data, and how to measure success, is a prerequisite for any successful application of machine learning, and it isn’t something that advanced tools like Keras and TensorFlow can help you with.”

“In general, anything that requires reasoning – like programming or applying the scientific method – long-term planning, and algorithmic data manipulation is out of reach for deep-learning models, no matter how much data you throw at them.”

“…deep-learning models don’t have any understanding of their input – at least, not in a human sense. Our own understanding of images, sounds and language is grounded in our sensorimotor experiences as humans. Machine-learning models have no access to such experiences and thus can’t understand their inputs in a human-relatable way…never fall into the trap of believing that neural networks understand the task they perform…Show them anything that deviates from their training data, and they will break in absurd ways.”

“Humans are capable of far more than mapping immediate stimuli to immediate responses, as a deep network, or maybe an insect, would. We maintain complex, abstract models of our current situation, of ourselves, and of other people, and can use these models to anticipate different possible futures and perform long-term planning…This ability to handle hypotheticals, to expand our mental model space far beyond what we can experience directly – to perform abstraction and reasoning – is arguably the defining characteristic of human cognition.”

“…don’t expect any singularitarian robot apocalypse to ensue: that’s pure fantasy, coming from a long series of profound misunderstandings of both intelligence and technology.”

“Don’t believe the short-term hype, but do believe in the long-term vision. It may take a while for AI to be deployed to its full potential…but AI is coming, and it will transform our world in a fantastic way.”

“…arXiv (https://arxiv.org/)…has become the de facto way to stay up to date on the bleeding edge of machine learning and deep learning.”

“Learning is a lifelong journey…”

Arrange a Conversation