Sources and Methods #44: Deep Learning with's Jeremy Howard


Jeremy Howard 101:

Jeremy on Twitter: JeremyPHoward

Free online programme / MOOC (“Practical Deep Learning for Coders”) at:

“The wonderful and terrifying implications of computers that can learn” (YouTube)

Show Notes:

5:55 - My entire education is one degree in philosophy. 

7:30 - Joined McKinsey at 18 with extremely basic knowledge.

12:19 - At our target audience really is people who have interesting and useful problems, and have a feeling that using AI might be a useful way to do that, that maybe don’t have a background in machine learning. It’s the people I came across in my career who were working in extremely diverse industries and roles and geographies, who are smart and passionate and working on interesting and important problems but don’t have any particular background in computer science or math. There’s a snobbish-ness in machine learning, that most people in it have extremely homogeneous backgrounds, young, white, male, who have studied computer science at a handful of universities in America or Europe. 

David Perkins at Harvard, and his learning theory of the ‘Whole Game.’ 

18:10 - For some reason, the STEM field on the whole have gotten away with shoddy, slack teaching methods, where we expect the students to do the work of sticking with it for 10 years and putting it all together. 

20:02 - We’ve discovered that the most practical component in AI is transfer learning. Taking a model that someone else has created and fine tuning it for your task. It turns out that this is the most important thing by far for actually getting AI to work in the real world. Apply and transfer learning effectively. 

I think many people teach a list or a menu of things that they know, rather than really getting to student learning. 

22:41 - Each year, we try to get to a point where the course covers twice as much as the previous year, with half as much code, with twice the accuracy at twice the speed. So far, we’ve been successful at doing that three years running. 

28:48 - I think that will be one of the two most important skills over the next decade or two - the idea of how to work as a domain expert to provide appropriate data to a machine learning system and to interpret the results of those things in a way appropriate to your work. If you don’t know how to do it, you’re going to be totally obsolete. 

31:09 - Back in the early days of the commercial internet, being an internet expert was extremely useful and you could have a job as an internet expert and be in a company of internet experts, and sell yourself as an internet expert company. Today, very few people do that, because on the whole the internet is what it is, and there’s a relatively few number of people who need such a level of expertise that they can go in and change the way your router operates and such. I think we’re going to see the same thing with AI. 

39:08 - I started learning Chinese not because I had any interest in Chinese, but because I was such a bad language learner in highschool. I did six months of French, I got 28% and I quit. When I wanted to dig into machine learning, I thought one of the things that might be better to understand was human learning, so I used myself as a subject. A hopeless subject. If I can come up with a way that even I can learn a language, that would be great. And to make sure that was challenging enough, I tried to pick the hardest language I could. So according to according to CIA guidelines, Arabic and Chinese are the hardest languages for people to pick up. Then I spent three months studying learning theory, and language learning theory, and then software to help me with that process. 

It turns out that even I can learn Chinese. After a year of this - by no means a full time thing, an hour or two a day - I went to China to a top language learning program and based on the results of my exam got placed with all these language PhDs, and I thought wow. Studying smart is important. It’s all about how you do it. 

Spaced repetition is such an easy thing that anyone can do, for free, you can start using it. 

[Jeremy’s amazing Anki talk]

If you’re not using Anki, you’re many orders of magnitude less likely to remember a piece of vocab. So you come away like I did, thinking you can’t learn a language. But once you learn vocab, the rest is really not that hard. Don’t try to learn grammar, just spend all your time reading. 

45:04 - If you’re not spending a significant portion of your early learning, learning how to learn, then you’re going to be at a disadvantage to those that did for that entire learning journey. Spending 12 years at school learning things, but nobody ever thought you how to learn, is the dumbest things I’ve ever heard. 

Coursera’s most popular course is Learning How To Learn

Exercise is the other most important thing. 

49:03 - My third superpower is taking notes. Exceptional people take a lot of notes. Less exceptional people assume they’re going to remember. 

50:19 - Taking notes in class is kind of a waste of time. I don’t really see the point of going to class most of the time honestly, it’s probably being videotaped. 

52:54 - Learn Python if you’re interested in data science, deep learning. 

54:22 - I think there are two critical skills going forward, pick one. One is knowing how to use machine learning. And the other is knowing how to interact with and care for human beings. Because the latter one can’t be replaced by AI. The former one will gradually replace everything.

Sources and Methods #26: Alex Mullen


Alex Mullen 101:

Alex’s Website

Alex’s Twitter Feed

Alex's YouTube channel

The Man Who Thinks Like Sherlock Holmes - BBC Profile


Show Notes:

3:14 (On where it all started) I read the techniques (in Joshua Foer’s book Moonwalking With Einstein) and I was hooked enough, and thought I really want to apply these to my learning life. A lot of people try to market these techniques as a way to improve your daily life, and there are some areas where you can use the techniques to learn things like languages. But to be honest, there’s not a whole lot of benefit. In my mind, it’s not like going from no techniques to using them has a whole lot of benefit to your daily life.

4:55 - My main interest is using these techniques for learning. I read the book, started practicing. I read a book by Dominic O’brien and took it forward from there.

8:25 - In terms of learning, it really took me a long time. I started in March 2013, when I was a junior in college, and it wasn’t until towards the end of my senior year that I started to apply the techniques and failed, and gave up on it, but came back to it when med school started, there’s so much stuff to memorize for med school. So it was nearing two years of exposure to the techniques before I felt that I was using them effectively.

11:02 - What you have to do if you want to make the techniques work is make them as simple as possible. Making memory palaces, struggling to convert information into images, it feels like it’s taking extra time.

12:14 - I wouldn’t say my natural memory has improved at all, it’s not like I remember where my car keys are all the time. But I would say my visualization skill has improved. Maybe not even in the sense of visualizing them more clearly, but more comfortable dipping into visualizing something. Being comfortable visualizing, that’s a transferrable skill.

14:03 (On Memorizing a deck of cards) I memorize card pairs and put them into an image. So I have over 1,000 images - random people, objects, characters from TV shows. So what I’m doing is seeing these people and those objects interact in the locus of the memory palace.

19:02 - Review, even if you’re using memory techniques, is pretty essential.

22:15 - One of the most important things is to not try to memorize everything. I think it’s a fairly common trap people run into.

24:00 - I use Anki pretty much for everything.

30:00 - For medical school, it’s all video podcasts for the most part, and what I do is take notes with Anki. And then I keep reviewing everything inside Anki from there on out.

34:40 - (On learning Chinese) I have a very systematic approach to this (in contrast to learning Spanish). The problem for me was that it’s too different from English. The idea, generally, is you take an English word that sounds like the foreign word and that’s your image for that foreign word. The problem with Chinese is you end up having… problems dealing with tonal nuances. The nice thing of Chinese is that since everything is one syllable, there’s a fixed number of ways you can start out a word and a fixed number of ways you can end that syllable. So the way it works is I have a character assigned to each of the starts, and then a place assigned to each of the ending sounds…. That sounds like a very complicated process, and it is, and it’s taken me a decent amount of practice to get fluent at doing that. I don’t know if I would’ve been up for that before doing competitions.

39:06 - One thing doing memory sports has definitely taught me is that a lot of barriers are psychological. It’s akin to Roger Bannister running the four minute mile for the first time, and within a short time, everyone was running four minute miles.

For me, trying to break plateaus has always been about trying to identify the key skill that needs to be improved, that’s holding you back. A lot of the time for me, for really anything system based, the key skill has been going as quickly as possible from information to an image. Realizing that so many things are psychological just changes your perspective.

44:04 - Learning how to learn is not really taught in school at all. That seems like a pretty huge oversight. Just giving kids the knowledge that you can imagine things like a giant sumo breaking down a tree, if that helps you learn, go for it.

48:43 - I think competitive sports is something that helps me. For most of my life, I was doing some kind of competitive sport, mainly swimming and tennis. I think that’s one of the main things that drew me to memory sports - like it or not, it is a sport, you’re training, you’re trying to get better at something, and the attraction of physical sports is still there with memory sports. Break milestones, it feels like a mental challenge you can work towards accomplishing.

50:39 - Every day I spent a few minutes running through a constricted deck of Anki cards for memory sports. That’s a daily training routine. For memory sports, I have a weekly schedule that I try to stick to. Usually it’s not much more than 30 minutes or so. For Spanish, I’m also using Duolingo. I’m also making my way through a Memrise course. Anki practice for medicine and memory sports. Those are my daily things.


Alex Mullen’s Book:

Moonwalking With Einstein by Joshua Foer

Matt’s Pick:

The Little Book of Talent by Daniel Coyle

Alex Mullen’s Film:


Alex’s Pick:

Harass Alex about learning Kanji

Alex Mullen’s Music:

The Brainfood Playlist on Spotify