Sources and Methods #44: Deep Learning with fast.ai's Jeremy Howard
Jeremy Howard 101:
Jeremy on Twitter: JeremyPHoward
Free online programme / MOOC (“Practical Deep Learning for Coders”) at: fast.ai
“The wonderful and terrifying implications of computers that can learn” (YouTube)
Show Notes:
5:55 - My entire education is one degree in philosophy.
7:30 - Joined McKinsey at 18 with extremely basic knowledge.
12:19 - At Fast.ai our target audience really is people who have interesting and useful problems, and have a feeling that using AI might be a useful way to do that, that maybe don’t have a background in machine learning. It’s the people I came across in my career who were working in extremely diverse industries and roles and geographies, who are smart and passionate and working on interesting and important problems but don’t have any particular background in computer science or math. There’s a snobbish-ness in machine learning, that most people in it have extremely homogeneous backgrounds, young, white, male, who have studied computer science at a handful of universities in America or Europe.
David Perkins at Harvard, and his learning theory of the ‘Whole Game.’
18:10 - For some reason, the STEM field on the whole have gotten away with shoddy, slack teaching methods, where we expect the students to do the work of sticking with it for 10 years and putting it all together.
20:02 - We’ve discovered that the most practical component in AI is transfer learning. Taking a model that someone else has created and fine tuning it for your task. It turns out that this is the most important thing by far for actually getting AI to work in the real world. Apply and transfer learning effectively.
I think many people teach a list or a menu of things that they know, rather than really getting to student learning.
22:41 - Each year, we try to get to a point where the course covers twice as much as the previous year, with half as much code, with twice the accuracy at twice the speed. So far, we’ve been successful at doing that three years running.
28:48 - I think that will be one of the two most important skills over the next decade or two - the idea of how to work as a domain expert to provide appropriate data to a machine learning system and to interpret the results of those things in a way appropriate to your work. If you don’t know how to do it, you’re going to be totally obsolete.
31:09 - Back in the early days of the commercial internet, being an internet expert was extremely useful and you could have a job as an internet expert and be in a company of internet experts, and sell yourself as an internet expert company. Today, very few people do that, because on the whole the internet is what it is, and there’s a relatively few number of people who need such a level of expertise that they can go in and change the way your router operates and such. I think we’re going to see the same thing with AI.
39:08 - I started learning Chinese not because I had any interest in Chinese, but because I was such a bad language learner in highschool. I did six months of French, I got 28% and I quit. When I wanted to dig into machine learning, I thought one of the things that might be better to understand was human learning, so I used myself as a subject. A hopeless subject. If I can come up with a way that even I can learn a language, that would be great. And to make sure that was challenging enough, I tried to pick the hardest language I could. So according to according to CIA guidelines, Arabic and Chinese are the hardest languages for people to pick up. Then I spent three months studying learning theory, and language learning theory, and then software to help me with that process.
It turns out that even I can learn Chinese. After a year of this - by no means a full time thing, an hour or two a day - I went to China to a top language learning program and based on the results of my exam got placed with all these language PhDs, and I thought wow. Studying smart is important. It’s all about how you do it.
Spaced repetition is such an easy thing that anyone can do, for free, you can start using it.
[Jeremy’s amazing Anki talk]
If you’re not using Anki, you’re many orders of magnitude less likely to remember a piece of vocab. So you come away like I did, thinking you can’t learn a language. But once you learn vocab, the rest is really not that hard. Don’t try to learn grammar, just spend all your time reading.
45:04 - If you’re not spending a significant portion of your early learning, learning how to learn, then you’re going to be at a disadvantage to those that did for that entire learning journey. Spending 12 years at school learning things, but nobody ever thought you how to learn, is the dumbest things I’ve ever heard.
Coursera’s most popular course is Learning How To Learn.
Exercise is the other most important thing.
49:03 - My third superpower is taking notes. Exceptional people take a lot of notes. Less exceptional people assume they’re going to remember.
50:19 - Taking notes in class is kind of a waste of time. I don’t really see the point of going to class most of the time honestly, it’s probably being videotaped.
52:54 - Learn Python if you’re interested in data science, deep learning.
54:22 - I think there are two critical skills going forward, pick one. One is knowing how to use machine learning. And the other is knowing how to interact with and care for human beings. Because the latter one can’t be replaced by AI. The former one will gradually replace everything.
Sources & Methods #41: Improving Counterterrorism with Stephen Tankel
Stephen Tankel 101:
Twitter: @StephenTankel
Reaching out: tankel@american.edu if people have questions.
Professor at American University - https://www.american.edu/sis/faculty/tankel.cfm
Senior Editor at War On The Rocks - War on the Rocks
Adjunct Senior Fellow at the Center for New American Security
First book: Storming the World Stage: The Story of Lashkar-e-Taiba
New Book: With Us and Against Us: How America's Partners Help and Hinder the War on Terror
Show Notes:
4:51 - What I was trying to do (with this book) is ask the question: If most of these partners are both helpful and hurtful, what can the United States reasonably expect from them? And then to offer propositions about what it can expect in terms of cooperation, with the understanding that managing expectations here and what to expect from partners here in knowing what to expect from partners is a critical component.
5:50 - The inspiration for the book came from the year I spent working in government as a senior advisor at the Department of Defense, working primarily in south and central asia, and being forced to wrestle from a policy perspective with the tradeoffs involved in dealing with these countries.
10:21 - I think it’s fair to say that the Bush Administration had often prioritized counter terrorism interests above other interest, and the Obama Administration tried to treat them as a separate entity rather than what really needs to happen in either of those administrations - which is integrating counter-terrorism within the broader US foreign policy approach.
11:03 - There’s another threat between short term security interests and long term interests in promoting good governance, rule of law, human rights, stability overall. We see this with a new report from a US Institute of Peace report on Fragile States.
It’s really hard to do, when you’re focused on shorter term objectives like access for forces for security cooperation, training and equipping local CT forces, or for drone strikes.
These tradeoffs are not unique to the United States.
13:12 - This book is primarily for US policy makers and people in those circles, and also intended to get me tenure (laughs).
17:21 - What I really try to do is get at things from the perspective of the partner with whom the United States is partnering. So I spend as much if not more time about the threat perceptions, the politics, the security compulsions of the partner in question than I do the United States.
It’s strategic empathy - I’m trying to get into the shoes of a partner.
18:09 - One of the main recommendations I have at the end is that the United States needs to devote at least as much time to understanding the perspectives and perceptions of its partners as it does its own internal machinations. Within the confines of the book, I’m trying to get at this from other countries’ perspectives.
23:33 - Another point on regionalism (to Matt’s question) - I think you’re talking primarily about regionalism from the perspective of having people with expertise on this regions. I think that is important. But I think there’s another type of regionalism - to create instruments of statecraft, policies, what we would call Congressional Authorities in the US, that are regional rather than nation specific, that encourage being able to work across a region or across part of a region from a policy perspective rather than always working bilaterally.
Bilateral is going to remain the primary mechanism through which any two countries. But at least from a US perspective, I have encouraged the idea of regional authorities for security assistance, cooperation, development, and things like that because I do believe it is helpful to take a regional view to these issues rather than working bilaterally.
26:22 - Quite frankly - standard metrics for me, that’s the brass ring. I would settle for metrics. I would settle for State Department having metrics, DoD having metrics, and NCTC having metrics, I would settle, from a USG perspective, every agency having metrics.
[On why we don’t have standard metrics everyone can look at and figure out where things are] I think it’s human nature, I think it’s bureaucracy, I think it’s those different theories. I would add more. First - it is my sense that practitioners often don’t have an appreciation for spending time and money on measurement because they want to just get out there and do it. And they see spending time and money on measurement as taking away from everything they could be throwing at the problem. I’ve been an evangelist for the idea that metrics will help you get more bang for your buck. I don’t want to spend my time and money measuring, I want to spend it doing. So you need to change the way you think. You need to think about measurement as intrinsic to whatever you’re trying to accomplish.
Two, I don’t think any bureaucratic culture plays to the strengths of monitoring and evaluation. Because monitoring and evaluation is meant to be objective. And objectively speaking, not every program is going to succeed. Simply because your program fails doesn’t mean it’s your fault. But nobody wants to be the person who was running the program that failed. So I do think there is a human nature issue but especially a bureaucratic culture issue that pushes back against monitoring and evaluation because nobody wants to be on the one who runs the program that doesn’t go well.
One of my dream projects that I want to find funding for is to explore ways in which it might be possible to import into a government culture the culture that in some ways, if not favors, at least s applauds failing early in Silicon Valley or business or something.
This idea that effective monitoring and evaluation - it shouldn’t be that you don’t want to fail, it should be that you want to fail early and figure it out so that you can reform. But that requires a big cultural change within government, within UN, within anything about how we think about these issues.
Monitoring and evaluation is is hard. It’s hard to gather data. There are disagreements about how to analyze that data.
Article at War on the Rocks: Doing More With Less: How to Optimize US Counter-terrorism
37:32 - One of the areas where we just don’t have a good sense of how well or poorly we’re doing is the question of resiliency. I’f you’d asked me 5-7 years ago, I would’ve said we’re doing poorly on that. Now, I just don’t know because we haven’t had a major attack. We’ve had some smaller attacks in the US but we’ve kind of gone about our business. At the end of the day, it may be policy makers who are in some cases - I don’t want to say more seriously than they should, but are inflating it more than it needs to be more than the general public.
39:30 - If one looks at where we ultimately want to go with this - it’s that this becomes for most of these countries a law enforcement problem and not a military problem, and that it is a problem that not just their police are strong enough to deal with, but their judiciaries are strong enough and they have prosecutorial capacity and they have capable judiciaries that are able to prosecute people that are involved in terrorism or terrorism-related offenses. And they have prisons that are capable of holding these people where they will not be radicalized. Those are really big asks.
42:06 - [On training police and justice systems actors vs training military soldiers in foreign countries] The United States, for legal reasons, has a lot of trouble training police. Because it used to be that the secret police were used to terrorize the population, so we have laws on the books going against training police. Those laws need to change.
45:40 - Individually, policy makers are all really smart. Collectively, policy-making does not look that smart.
Even though individuals may not be risk averse, institutions typically are risk averse.
57:30 - [On Useful Tools for Your Career] Learning Languages - I lived in Egypt for awhile, I studied in Syria, I spent a lot of time for my PhD on the ground in Algeria and Lebanon, using other language. I think it’s not only obviously useful in being able to conduct an interview or read a newspaper, two things I would probably struggle to do nearly as well today because I haven’t used it as much as I should have, but the simple - and it’s hardly simple - the exercise of trying to learn a language in and of itself is helpful in understanding other people, other cultures, what have you. I think there’s a lot of value in having the exercise of trying to learn, even if you’re never going to be that great of a linguist. I often encourage my students who have an interest in another part of the world to go live there. I think living in other places - its a bit trite - but it’s an eye-opening experience. I think it’s very easy to say, much harder to do.
1:00:13 - To say to people coming up - ultimately at the end of the day, there’s no substitute for this other than doing it for awhile, which is something people told me 10 years ago when I was starting out and I found frustrating, but now a decade out, I find useful.
And that was useful for government - I understood it a lot better after being inside it.
1:01:40 - Advice from Vali Nasr - He said, Stephen, if you learn a single thing about South Asia during your year in government, you can no longer call yourself an expert on South Asia. Your job is to be a participant observer. It is to work on whatever your bosses want you to work on. It is to participate in the bureaucracy. Go think big strategic thoughts whenever they want you to, but really learn how everything works so that when you come out you have an understanding of the challenges the average bureaucrat faces. That’s an approach I’ve tried to take with me with everything.