Sources and Methods #44: Deep Learning with fast.ai's Jeremy Howard

jh-head.jpg

Jeremy Howard 101:

Jeremy on Twitter: JeremyPHoward

Free online programme / MOOC (“Practical Deep Learning for Coders”) at: fast.ai

“The wonderful and terrifying implications of computers that can learn” (YouTube)

Show Notes:

5:55 - My entire education is one degree in philosophy. 

7:30 - Joined McKinsey at 18 with extremely basic knowledge.

12:19 - At Fast.ai our target audience really is people who have interesting and useful problems, and have a feeling that using AI might be a useful way to do that, that maybe don’t have a background in machine learning. It’s the people I came across in my career who were working in extremely diverse industries and roles and geographies, who are smart and passionate and working on interesting and important problems but don’t have any particular background in computer science or math. There’s a snobbish-ness in machine learning, that most people in it have extremely homogeneous backgrounds, young, white, male, who have studied computer science at a handful of universities in America or Europe. 

David Perkins at Harvard, and his learning theory of the ‘Whole Game.’ 

18:10 - For some reason, the STEM field on the whole have gotten away with shoddy, slack teaching methods, where we expect the students to do the work of sticking with it for 10 years and putting it all together. 

20:02 - We’ve discovered that the most practical component in AI is transfer learning. Taking a model that someone else has created and fine tuning it for your task. It turns out that this is the most important thing by far for actually getting AI to work in the real world. Apply and transfer learning effectively. 

I think many people teach a list or a menu of things that they know, rather than really getting to student learning. 

22:41 - Each year, we try to get to a point where the course covers twice as much as the previous year, with half as much code, with twice the accuracy at twice the speed. So far, we’ve been successful at doing that three years running. 

28:48 - I think that will be one of the two most important skills over the next decade or two - the idea of how to work as a domain expert to provide appropriate data to a machine learning system and to interpret the results of those things in a way appropriate to your work. If you don’t know how to do it, you’re going to be totally obsolete. 

31:09 - Back in the early days of the commercial internet, being an internet expert was extremely useful and you could have a job as an internet expert and be in a company of internet experts, and sell yourself as an internet expert company. Today, very few people do that, because on the whole the internet is what it is, and there’s a relatively few number of people who need such a level of expertise that they can go in and change the way your router operates and such. I think we’re going to see the same thing with AI. 

39:08 - I started learning Chinese not because I had any interest in Chinese, but because I was such a bad language learner in highschool. I did six months of French, I got 28% and I quit. When I wanted to dig into machine learning, I thought one of the things that might be better to understand was human learning, so I used myself as a subject. A hopeless subject. If I can come up with a way that even I can learn a language, that would be great. And to make sure that was challenging enough, I tried to pick the hardest language I could. So according to according to CIA guidelines, Arabic and Chinese are the hardest languages for people to pick up. Then I spent three months studying learning theory, and language learning theory, and then software to help me with that process. 

It turns out that even I can learn Chinese. After a year of this - by no means a full time thing, an hour or two a day - I went to China to a top language learning program and based on the results of my exam got placed with all these language PhDs, and I thought wow. Studying smart is important. It’s all about how you do it. 

Spaced repetition is such an easy thing that anyone can do, for free, you can start using it. 

[Jeremy’s amazing Anki talk]

If you’re not using Anki, you’re many orders of magnitude less likely to remember a piece of vocab. So you come away like I did, thinking you can’t learn a language. But once you learn vocab, the rest is really not that hard. Don’t try to learn grammar, just spend all your time reading. 

45:04 - If you’re not spending a significant portion of your early learning, learning how to learn, then you’re going to be at a disadvantage to those that did for that entire learning journey. Spending 12 years at school learning things, but nobody ever thought you how to learn, is the dumbest things I’ve ever heard. 

Coursera’s most popular course is Learning How To Learn

Exercise is the other most important thing. 

49:03 - My third superpower is taking notes. Exceptional people take a lot of notes. Less exceptional people assume they’re going to remember. 

50:19 - Taking notes in class is kind of a waste of time. I don’t really see the point of going to class most of the time honestly, it’s probably being videotaped. 

52:54 - Learn Python if you’re interested in data science, deep learning. 

54:22 - I think there are two critical skills going forward, pick one. One is knowing how to use machine learning. And the other is knowing how to interact with and care for human beings. Because the latter one can’t be replaced by AI. The former one will gradually replace everything.

Sources & Methods #41: Improving Counterterrorism with Stephen Tankel

 
Stephen-Tankel.jpg
 

Stephen Tankel 101:

stephentankel.com

Twitter: @StephenTankel

Reaching out: tankel@american.edu if people have questions.

Professor at American University -  https://www.american.edu/sis/faculty/tankel.cfm

Senior Editor at War On The Rocks - War on the Rocks

Adjunct Senior Fellow at the Center for New American Security

First book: Storming the World Stage: The Story of Lashkar-e-Taiba
New Book: With Us and Against Us: How America's Partners Help and Hinder the War on Terror

Show Notes:

4:51 - What I was trying to do (with this book) is ask the question: If most of these partners are both helpful and hurtful, what can the United States reasonably expect from them? And then to offer propositions about what it can expect in terms of cooperation, with the understanding that managing expectations here and what to expect from partners here in knowing what to expect from partners is a critical component.

5:50 - The inspiration for the book came from the year I spent working in government as a senior advisor at the Department of Defense, working primarily in south and central asia, and being forced to wrestle from a policy perspective with the tradeoffs involved in dealing with these countries.

10:21 - I think it’s fair to say that the Bush Administration had often prioritized counter terrorism interests above other interest, and the Obama Administration tried to treat them as a separate entity rather than what really needs to happen in either of those administrations - which is integrating counter-terrorism within the broader US foreign policy approach.

11:03 - There’s another threat between short term security interests and long term interests in promoting good governance, rule of law, human rights, stability overall. We see this with a new report from a US Institute of Peace report on Fragile States.

It’s really hard to do, when you’re focused on shorter term objectives like access for forces for security cooperation, training and equipping local CT forces, or for drone strikes.

These tradeoffs are not unique to the United States.

13:12 - This book is primarily for US policy makers and people in those circles, and also intended to get me tenure (laughs).

17:21 - What I really try to do is get at things from the perspective of the partner with whom the United States is partnering. So I spend as much if not more time about the threat perceptions, the politics, the security compulsions of the partner in question than I do the United States.

It’s strategic empathy - I’m trying to get into the shoes of a partner.

18:09 - One of the main recommendations I have at the end is that the United States needs to devote at least as much time to understanding the perspectives and perceptions of its partners as it does its own internal machinations. Within the confines of the book, I’m trying to get at this from other countries’ perspectives.

23:33 - Another point on regionalism (to Matt’s question) - I think you’re talking primarily about regionalism from the perspective of having people with expertise on this regions. I think that is important. But I think there’s another type of regionalism - to create instruments of statecraft, policies, what we would call Congressional Authorities in the US, that are regional rather than nation specific, that encourage being able to work across a region or across part of a region from a policy perspective rather than always working bilaterally.

Bilateral is going to remain the primary mechanism through which any two countries. But at least from a US perspective, I have encouraged the idea of regional authorities for security assistance, cooperation, development, and things like that because I do believe it is helpful to take a regional view to these issues rather than working bilaterally.

26:22 - Quite frankly - standard metrics for me, that’s the brass ring. I would settle for metrics. I would settle for State Department having metrics, DoD having metrics, and NCTC having metrics, I would settle, from a USG perspective, every agency having metrics.

[On why we don’t have standard metrics everyone can look at and figure out where things are] I think it’s human nature, I think it’s bureaucracy, I think it’s those different theories. I would add more. First - it is my sense that practitioners often don’t have an appreciation for spending time and money on measurement because they want to just get out there and do it. And they see spending time and money on measurement as taking away from everything they could be throwing at the problem. I’ve been an evangelist for the idea that metrics will help you get more bang for your buck. I don’t want to spend my time and money measuring, I want to spend it doing. So you need to change the way you think. You need to think about measurement as intrinsic to whatever you’re trying to accomplish.

Two, I don’t think any bureaucratic culture plays to the strengths of monitoring and evaluation. Because monitoring and evaluation is meant to be objective. And objectively speaking, not every program is going to succeed. Simply because your program fails doesn’t mean it’s your fault. But nobody wants to be the person who was running the program that failed. So I do think there is a human nature issue but especially a bureaucratic culture issue that pushes back against monitoring and evaluation because nobody wants to be on the one who runs the program that doesn’t go well.

One of my dream projects that I want to find funding for is to explore ways in which it might be possible to import into a government culture the culture that in some ways, if not favors, at least s applauds failing early in Silicon Valley or business or something.

This idea that effective monitoring and evaluation - it shouldn’t be that you don’t want to fail, it should be that you want to fail early and figure it out so that you can reform. But that requires a big cultural change within government, within UN, within anything about how we think about these issues.

Monitoring and evaluation is is hard. It’s hard to gather data. There are disagreements about how to analyze that data.

Article at War on the Rocks: Doing More With Less: How to Optimize US Counter-terrorism

37:32 - One of the areas where we just don’t have a good sense of how well or poorly we’re doing is the question of resiliency. I’f you’d asked me 5-7 years ago, I would’ve said we’re doing poorly on that. Now, I just don’t know because we haven’t had a major attack. We’ve had some smaller attacks  in the US but we’ve kind of gone about our business. At the end of the day, it may be policy makers who are in some cases - I don’t want to say more seriously than they should, but are inflating it more than it needs to be more than the general public.

39:30 - If one looks at where we ultimately want to go with this - it’s that this becomes for most of these countries a law enforcement problem and not a military problem, and that it is a problem that not just their police are strong enough to deal with, but their judiciaries are strong enough and they have prosecutorial capacity and they have capable judiciaries that are able to prosecute people that are involved in terrorism or terrorism-related offenses. And they have prisons that are capable of holding these people where they will not be radicalized. Those are really big asks.

42:06 - [On training police and justice systems actors vs training military soldiers in foreign countries] The United States, for legal reasons, has a lot of trouble training police. Because it used to be that the secret police were used to terrorize the population, so we have laws on the books going against training police. Those laws need to change.

45:40 - Individually, policy makers are all really smart. Collectively, policy-making does not look that smart.

Even though individuals may not be risk averse, institutions typically are risk averse.  
57:30 - [On Useful Tools for Your Career] Learning Languages - I lived in Egypt for awhile, I studied in Syria, I spent a lot of time for my PhD on the ground in Algeria and Lebanon, using other language. I think it’s not only obviously useful in being able to conduct an interview or read a newspaper, two things I would probably struggle to do nearly as well today because I haven’t used it as much as I should have, but the simple - and it’s hardly simple - the exercise of trying to learn a language in and of itself is helpful in understanding other people, other cultures, what have you. I think there’s a lot of value in having the exercise of trying to learn, even if you’re never going to be that great of a linguist. I often encourage my students who have an interest in another part of the world to go live there. I think living in other places - its a bit trite - but it’s an eye-opening experience. I think it’s very easy to say, much harder to do.

1:00:13 - To say to people coming up - ultimately at the end of the day, there’s no substitute for this other than doing it for awhile, which is something people told me 10 years ago when I was starting out and I found frustrating, but now a decade out, I find useful.

And that was useful for government - I understood it a lot better after being inside it.
1:01:40 - Advice from Vali Nasr - He said, Stephen, if you learn a single thing about South Asia during your year in government, you can no longer call yourself an expert on South Asia. Your job is to be a participant observer. It is to work on whatever your bosses want you to work on. It is to participate in the bureaucracy. Go think big strategic thoughts whenever they want you to, but really learn how everything works so that when you come out you have an understanding of the challenges the average bureaucrat faces. That’s an approach I’ve tried to take with me with everything.

Sources and Methods #31: J. Kael Weston

Alex’s PhD Tool Recommendations

Spaced Repetition Foundation

J. Kael Weston’s Website

JK Weston’s email for readers of his book, loves to hear from you: jkwestonmt@gmail.com

Show Notes:

7:00 - Joined the foreign service after failing out of my PhD program

10:48 - How do you do it [implement US foreign policy] on the ground? You call a lot of audibles. It’s all about relationships.

17:16 - The farther you are from a place like Fallujah or the farther you are from Anbar, Baghdad, Basra, you’re going to be dominated by domestic considerations matter more than what’s going on in the ground. I say this in the book - American elections matter more than any Iraqi or Afghan election, and I believe that.

20:15 - I was always trying to say that we’re not here to occupy your country, that we’re not here to suck out all your oil. The Marines are here to fight, and they’ll fight hard. And I’m here to say there’s a better way forward, where the troop levels go down. And I had another message for Washington - that even a guy here full time, I only knew 1%. Year by year I knew 1%.

21:40 - It behooves us as a country, even as a superpower, how little we can learn in a seven month tour, or a year tour, or even a three year tour.

24:10 My book is an exercise in resurrection in words, I’m trying to bring back in words, real characters, who deserve at least a few pages in a 600 book. And it’s got applications for today. I didn’t want to write a book with five policy lessons from Fallujah. What I wanted to do was write a book so that if you’re a SEAL in Coronado or your General Dunford or whether you’re just an average concerned citizen, you’ll understand that if you take an Iraqi female, there are going to be repercussions. And in the two longest wars in our history, have we learned those lessons and repercussions?

27:15 - I did not enter Fallujah looking at the world there in black and white at all. The two colors were grey and red. Way too much red, and grey everywhere. Some of my military colleagues, and I don’t fault them, had a harder time with that. If you were X then and you want to be Y now, that can’t work.

28:40 - When you listen to the people you’re there to interact with, they can tell you what you want to hear. It’s not as if they don’t want to come to some kind of arrangement with you. And I saw that in guys in the third or fourth tours who got that better…. I found that some of our best allies were folks who had been fighting us a year or two before, because they saw how bad things could get.

30:13 - War is about pragmatism on the ground. You can’t get on your high horse and say well, that guy was a bad guy three years ago. You can make a point, but you’ll never end a war if you don’t reach out and find more stability with more people.

31:39 - Warfare is about people. It’s about the cost to people in a war. People don’t win in a war. Never. Countries may win in a war, but never people. The people in Fallujah are still losing.

36:50 - In these wars we confused troop levels with commitment.

39:00 - I think we escalated a war, unnecessarily. We escalated in terms of troop levels - it’s like a big inverted V. We went up quickly and then the temptation was to  drop quickly. And what I would’ve liked to have seen, which I think would have allowed for more consistent people to people interaction, is to go low to go long. I’ve been advocating that for over a decade.

42:43 - We should be wary of where our limits. One of the best lessons to come from Afghanistan and Iraq is we have limits. The American superpower - which we didn’t see in 2003. I remember when we were at the height of our national power, people forget it was before the mortgage meltdown, before everything we went through, our troops on the fourth, fifth, sixth tour, we were all muscle at the time. Twitch muscle. And what we now see is we have limits. I think this [understanding] will eventually make us a stronger power, a more responsible power. I also think that focusing on the homeland alone is also a recipe for disaster.

52:42 - The problem (on sending people with language skills to countries they are working on) was urgency, and I hope that’s a lesson that’s learned. When you’re sending 150,000 soldiers to Iraq, this isn’t a priority. I don’t believe that the best representatives of the United States always are the experts in the region because at some level there’s a distance that’s helpful as well. I went native with the Sunnis in the West (of Iraq) and the Marine Corps, and I don’t think that undermined what we were doing, but you need a balance.

54:55 - The best diplomats are the ones who speak the language, know the history, and know the arc of that part of the world, for sure. And Amb. Crocker, Ford, and I could name a number of them, they’re out there.

55:55 - It’s a resurrection book. It’s also an accountability book. I believe that some of the literature out there is missing the lowercase ‘p’ for political frame. that wars get started and decisions are made and we shouldn’t just pretend it’s about me and my platoon. That’s an important story. But there’s also the story about the United States of America going to war. And because of my job I felt like I had curtains that I could pull back, in as an objective way I could, and in the best writing I could. So it’s not a counterinsurgency 501 lessons learned book, it’s - these are the stories, these are the people.

56:54 - The mechanics are two shoeboxes full of audiotapes. I used to speak into tape recorders in Iraq. I didn’t even rely on these as much as I could because I almost had too much information. So one day I may do another kind of book. The audio. You know, one day Sheikh Hamza is killed, I’m venting into an audio recorder and you can hear the call to prayer in the background in the middle of Fallujah and that’s all incredible audio. And it’s also in terms of writing where the details came from.

And  I also had a pile of those military green notebooks.

59:15 - The best ‘Mirror Test’ from these wars will come from them [the Iraqis and Afghans]. And I can’t wait until there’s an archive of - I hope one day - the Fallujans writing about the Americans in their city. And I might be able to help that. I hope that someday we’re reading the raw, unfiltered voices of the war. Because I’m still a filter, I still have my biases, I still made my decision on what to focus on. That’s why I ended the book with the soldier’s journal, because I wanted just his voice to be raw and pure, how it is when you’re in the middle of Sadr City. And why I end with the friends and family for the Marines who were killed in that helicopter crash.  

1:00:47 - I know many veteran writers who are very disciplined people who write 1,000 words a day. I’m not one of those people. If I’m motivated and the coffee is flowing, I may write a bit. I spend most of my time thinking about what I’m going to write. This book was actually in formation in my mind between National Parks out West because I needed to start thinking about how to dig into my seven year archive. I had too much information. I had too many stories.

1:02:40 - I still think the best books of these wars are yet to come.

For a non-fiction writer, I sought a fiction editor. I was searching for my weaknesses.

On how to launch a career like his:

1:06:07 - I would say the State Department is doing a much better job of looking like the rest of America. It’s not all Pale Male from Yale. And that’s good because the world needs to see us in all of our diversity and our strengths and weaknesses.

1:06:41 - I also would say that there’s no set game plan to pass the exam [Foreign Service Exam]. I don’t think you can just read every New York Times paper that comes out and start with the culture section and finish with A1. Read widely, go overseas, I’m always an advocate when you’re out of college - go see the world. You’re only going to be able to understand America by living the rest of the world. I believe that fundamentally. You’ll only be able to represent the United States well if - not just over a summer, or a two week tour to Venezuela or Italy or Japan - but to really cut yourself off from your country for awhile. And a lesson in so many ways. And then by the time you’ve had that experience - whether it’s a language program, Peace Corps, military - you will become the better diplomat, because you will know what our country looks and sounds like, oceans away. And it’s that empathy thing, you now know how they see us because you’re among. The good and the bad.

The best ambassadors we have are not our official ambassadors in our embassies. They are the ambassadors in the generation that say, Ok, the Americans still represent this, even though that.

[We find our greatest safety, we find the greatest stability when] We’re that leader that people want to follow not because they’re fearful of us, not because of our military.

So yes, go overseas, go to a hard place. Go to places that maybe are going to challenge you to your core. And then if you want to join the State Department or USAID and they ask you the question ‘so why are you qualified and or what do you want to do’ - that’s your best opportunity. Is to say here’s what I lived, here’s what I experienced.

 

Kael’s Picks:

Television

1973/74 British Television Series ‘World at War.’ Episode 18. Narrated by Lawrence Olivier. This episode, titled ‘Occupation Holland’ has moved me more than anything I’ve ever seen in my whole life. They focus on the Dutch experience.

Music

Love Vigilantes New Order. For service members, it will be incredibly moving.

Book

War Comes to Garmser by Carter Malkasian