Today I Learned

For the past two years, I’ve been working at one of the best educational technology companies in the world: Duolingo. I want to start writing about the things I’m learing about educational technology and I needed a space to do so. So here it is.

  1. May 09, 2021

    Since deciding to focus Grail Diary on book notes, I’ve changed the app’s navigation model to be reading-focused instead of notes-focused.

    screenshot

    Next up is extending the Review Mode. Currently review mode is all about spaced repetition and active recall. However, part of the value of having your book notes all in one place is perusability — I want a review mode that makes it easy to revisit your favorite quotes. What I implemented is a mode that quizzes you on the material you are likely to forget. (I still want a spaced repetition mode! I just also want another one that’s about revisiting your favorite quotes.)

    Finally, it’s time to look for an audience for Grail Diary. I’ve long been conflicted about if I should put Grail Diary on the App Store. I don’t want to turn Grail Diary into a side hustle. That doesn’t sit right with me for some reason. However, this past week I re-read a post by Brent Simmons where he writes, “This is the age of writing iOS apps for love.” That struck a chord. I’ve been working on this app for years because I love books, I love writing apps, and Grail Diary makes my experience of reading better. Writing an app for love why I’ve decided it’s worth the work of finding an audience for Grail Diary and putting it on the app store. I know there won’t be a huge audience for this app, but I also know there is a niche somewhere. And just like my writing gets better when I get feedback, Grail Diary will become better when I find the right audience and start getting feedback. That’s my goal.

  2. April 30, 2021

    I last wrote:

    When I was working on Grail Diary, I was confused on if I was writing a digital Commonplace Book (specifically designed for storing quotes and other things you want to remember about the books you read) or if I was writing a general-purpose notes app with a spaced-repetition feature…

    I’ve been thinking about this a lot for the past few days, and I’ve made a decision: Grail Diary is going to be an app for taking notes about the books you read. That’s currently what I use it for, and I want to start evolving the app to make it as great as possible for any other book lover.

    Book notes are important enough to deserve a dedicated application! For a book lover, your book notes will be among the most important things you create in your life. Your notes help books become a part of you.

    I designed Grail Diary around three factors that make book notes different:

    • Permanent value: If you are a book lover, you can build up a reading log and notes over the course of decades. My log and notes go back over twenty years. Grail Diary uses simple plain-text markup so your notes will be readable forever, by almost any application.
    • Personal ownership: Your book notes are yours. You shouldn’t lose them when a software company goes out of business. Grail Diary works with simple files! No account, no sign-up, and you can move or copy your files wherever you want. You can use file synchronization services like iCloud Drive or Dropbox to keep your content in sync across multiple devices.
    • Perusability: The joy of writing book notes is rereading them! It’s like meeting old friends again. Grail Diary already has features to help you get reacquainted with what you’ve put in your notes, and I’ve got ideas for many more.

    Even if Grail Diary never makes it beyond “personal project,” I feel better with focus. I’m trying to decide if I want to really polish the app and put it on the app store, or if I just want to keep it as an open-source app for the motivated techies to benefit from. On the one hand, getting my own app back on the app store would be a good ultralearning project. On the other hand, I’m kind of scared to take that step. If I release the app to the app store and people don’t like it… can my ego handle it?

    That’s a decision for another day.

  3. April 25, 2021

    Three years ago today, I was reading the book Factfulness by Hans Rosling, and I wrote in my journal that I’d like a program that would add Anki-like spaced repetition to the notes I was making about the book. I wrote:

    I’m thinking of maintaining a simple text file with Markdown syntax, one bulleted line per “fact” I want to remember from a book. Markdown-underline something like this and it becomes a phrase that gets elided for an Anki card.

    That idea turned into Grail Diary, which I still use today to take notes about the books I read. The project has been a huge personal success. I write personal programs as a way to teach myself things, and by working on Grail Diary I cemented knowledge into my brain about piece tables, incremental packrat parsing, spaced repetition, sqlite, and iCloud document storage. I also have 760 prompts about the 66 books I’ve read in the past 3 years, and by regularly reviewing those cards I’ve remembered the material I’ve read these past three years way better than what I’d read for the prior 44. It was also the start of my journey into educational technology, which lead me to leave Facebook and join Duolingo.

    Of all of my side projects, Grail Diary feels like it’s got the most potential to be useful for someone other than myself. However, it’s got one huge problem at its core: When I was working on Grail Diary, I was confused on if I was writing a digital Commonplace Book (specifically designed for storing quotes and other things you want to remember about the books you read) or if I was writing a general-purpose notes app with a spaced-repetition feature. As a result, it’s this strange mishmash of features. I doubt anyone else would understand why the software in its current form behaves the way it does.

    Who knows… maybe by the time Grail Diary turns 5, I’ll have picked “Commonplace Book” or “General-purpose notes app” as the primary identity for the project and it will have another user. Time will tell!

  4. April 24, 2021

    I’m awed by Scott Young’s MIT Challenge for its simplicity and audacity. In 2011, he gave himself one year to complete the full four-year curriculum in Computer Science from MIT. MIT made most of its course material available for free online, including the tests and the answer keys, so Young could work at his own pace, at his own home, and not spend any tuition money on this experiment. He successfully finished this project in 2012. In 2019, he published the book Ultralearning, a book that helps people plan big learning projects of their own.

    Young writes that for any ultralearning project, you should budget about 10% of your time on metalearning — making a plan to identify what you need to learn and how you will go about learning it. Furthermore, he advises you to break down the what you need to learn into three buckets:

    1. New concepts that you need to understand
    2. New facts that you need to memorize
    3. New skills that you need to practice and acquire

    You do this because the techniques to efficiently learn things are different for the different categories. I’m shocked I’d never thought about this before! After reading this, I understand my own learning shortcomings much better than I did before. I love love love learning that falls in “understanding new concepts” category. I gravitated to subjects like math, physics, and computer science that are rich in first-principles conceptual understanding. However, the learning tools that help me pick up new concepts don’t help me pick up new skills or memorize things, so I struggled in subjects like foreign languages and art. I wish I’d had this book back in my high school years to know that I needed to use different tools to learn different things.

    Ultralearning also makes another important argument. As much as possible, you should structure your learning project around doing the thing you’re trying to learn how to do and you need a way to get feedback on how you are doing. Do you want to learn a language so you can speak to locals when you travel? Then you should be speaking to locals as much as you can as early as you can. (The reaction of the native speakers gives you real-time feedback!) If you want to learn jazz guitar, you need to spend a lot of time playing guitar.

    The book devotes a few pages arguing against the effectiveness of my employer, Duolingo, because it is a very indirect way to learn a language. At the same time, though, Young writes of the importance of using drills to isolate and improve specific skills for your learning project. Someone trying to improve at tennis will do more than play games; forehand / backhand / serving drills will help you isolate and improve the building block skills for the game faster than you can just from games. Duolingo plays a similar role for serious language learners. It’s not a substitute for talking to native speakers, but the app does help you drill on vocabulary and grammar. (Also! Duolingo provides way more than app-based translation exercises. You can use https://events.duolingo.com to find groups to practice speaking and listening. You can use Duolingo Podcasts for practice understanding native speakers. And perhaps most importantly, Duolingo offers learners of all levels motivation to keep learning — the hardest part of learning a new language.)

    Anyone who is interested in learning, and in particular anyone who is interested in self-directed learning, should read Ultralearning. You’ll find a ton of helpful material. For those interested in educational technology, Ultralearning suggests two things where technology seems uniquely positioned to help people learn faster and better: In providing material to practice and in feedback. All of the influential educational software I can think of — from Duolingo to Anki to Kahn Academy to experimental efforts like the “mnemonic medium” — deliver in both of these dimensions. However, I think because Ultralearning already assumes its reader is highly motivated to learn, it doesn’t say much about one of the most interesting contributions of educational technology. Successful technology makes learning fun and contains mechanisms to help sustain motivation over time.

  5. April 21, 2021

    Since I lamented about the black-box nature of performance engineering in SwiftUI two days ago, I spent some time familiarizing myself with the SwiftUI tools inside of Instruments. While I’ve made some headway, I’ve also hit a wall.

    First, some context. As I mentioned earlier, Captain’s Log is a simple habit-tracking app that I want to use as a playground for experimenting with streaks, streak freezes, and the psychology of motivation. It’s currently a document-based app that stores its data in a plain text file. Right now there are only two screens. The main screen shows a day’s status on each habit and a calendar to help visualize how each streak is going. Then, there’s a second screen to tweak any additional details for completing a habit. (For example, to prepare for exiting quarantine, I’m trying to ride my bike at least a little every day. When I record a bike ride in Captain’s Log, I track how long and how far I ride.)

    Streak Visualization

    While performance isn’t terrible, there are noticeable lags in several interactions. For the past two evenings, I wanted to eliminate one of the lags: There is a noticeable delay processing keystrokes in the edit form, with the first keystroke being the worst.

    Here’s what I learned after a few days poking around with the SwiftUI tools in Instruments.

    1. My pre-SwiftUI Instruments workflow of just looking at heaviest stack frame in Time Profiler and optimizing that doesn’t work in this case. The heaviest stacks are all deep in SwiftUI library code that I don’t understand.
    2. When debugging UI glitches, the Core Animation tool is really helpful. All of the places where I noticed that the UI was lagging, like typing characters, were visible in the Core Animation track as “long transaction commits.” For instance, before any performance optimizing, there’d be a 135ms core animation commit when processing the first keystroke in my edit form. Having these sections called out in the track let me focus specifically on what was happening at this problematic times.
    3. Paul Hudson pointed out that you can use the Hide System Libraries option in Time Profiler to quickly find bottlenecks in your code instead of the SwiftUI code. This helped! I found a couple of places where I was doing excessive calendar math when drawing the streak visualization view. However, unnecessary calendar math was only about 25% of the CPU use during the long transaction commits — the rest is SwiftUI library code. With my optimizations I got the long commit down to 100ms. Better, but still way too long for processing a keystroke.
    4. The SwiftUI View Body tool showed that my view bodies aren’t that heavy. Most compute in 2-4 microseconds. In the span of a 100ms core animation commit, I spend 0.6ms computing view bodies for my own views and a total of 2ms computing all view bodies. 98% of the time is spent somewhere else.
    5. But here’s what I don’t understand. The SwiftUI View Properties tool shows that my main FileDocument struct changes on each keystroke. And I assume because the FileDocument changes, SwiftUI recomputes everything that depends on that document (basically, all the UI in the app). On every keystroke. I don’t understand this at all. Inspecting my code, it doesn’t look like the file document should be changing on each keystroke (the text fields are backed by a separate String binding independent of the FileDocument until you tap Done). I wrote some custom Binding code and set breakpoints everywhere I could think to validate that the document is not changing on each keystroke. In spite of that, SwiftUI is convinced that it needs to recompute everything that depends on this document every time I enter a new character in a text field.

    I don’t know how to debug this further. This is exactly what I meant when I wrote earlier about the inherent tension between declarative UI frameworks and performance tuning. I’ve described what I want in my UI (“a monthly calendar with line segments showing how long I’ve maintained different streaks”). There are probably things I can do to make that “what” even more efficient. However, code I don’t understand and has don’t have access to has decided the “how” of making my app work involves recomputing that calendar on each keystroke. I don’t know how I can make the app responsive without having the ability to influence that “how.”

    Since Captain’s Log is a toy app meant for me to learn, I’m just going to leave things as is and hope that Apple provides better performance guidance and tools at WWDC 2021.

  6. April 19, 2021

    Yesterday, I wrote about streaks and motivation. To let me experiment with streaks and streak freezes, I’ve started work on a simple habit-tracking app. One of my personal goals is to become proficient in SwiftUI, so I used SwiftUI for this project.

    The good news: My project, Captain’s Log, is done “enough” for me to use it. It’s also pleasingly compact (1200 lines of code). However, I’m now wrestling with performance. This is the slowest app I’ve written in a long time, and in spite of working as an iOS performance engineer at Facebook for years, I have no idea how to make this simple program faster. The time profiler tool in Instruments shows me deep, incomprehensible stacks that have nothing obvious to do with my SwiftUI code. The new View Body and View Properties tools are a little more helpful. For instance, one performance problem I have is the app takes too long to process keystrokes. Using the new tools, it looks like my central document property updates on each keystroke, and this causes most of the app to redraw. However, I can’t figure out why this property is updating on each keystroke, nor can I tell if I’ve broken some intelligent View diffing that’s supposed to be happening. I feel stuck.

    When Apple introduced SwiftUI, they explained the difference between imperative and declarative programming with a sandwich shop metaphor. If you walk into a sandwich shop and say, “I’d like an avocado toast,” that’s like declarative programming. You’re describing what you want and you let the server figure out how to make it. To get an avocado toast imperative-style, you’d need to tell the server individual steps instead. (“First, I want you to get a slice of bread. Next, toast it for 2 minutes. Then, get a properly ripe avocado. Mush some avocado and spread it on the toast…“)

    I love this metaphor! It shows the promise of declarative frameworks — and also hints at why performance problems might be inherently harder to solve with them. Suppose I order an avocado toast at brunch, and the server disappears. 20 minutes pass. 30 minutes. Where’s my food? Since I don’t know the steps that the server takes to fulfill my order, there’s no way to figure out why things are taking so long. This seems to be the state of performance tuning in SwiftUI: You, sitting at a table, alone & hungry, wondering where your food is.

    Clearly, if I’m going to become proficient with SwiftUI, I’m going to need to learn some new performance skills. Paul Hudson has the best performance tuning guide I’ve found so far, and my next project is to see if I can use this to make Captain’s Log pleasantly snappy.

    Always new skills to learn!

  7. April 18, 2021

    Inside Duolingo, we have a saying: The hardest part about learning a new language is staying motivated. I didn’t appreciate this aspect of effective educational technology before I started working here. The best educational software will not only have great content: It will have mechanisms that help learners stay motivated to keep learning.

    Streaks are one of the most important mechanisms that Duolingo uses to keep people motivated. Streaks encourage people to do an activity a little bit every day by counting the number of consecutive days you’ve done something you care about (spent time studying a language, got some exercise, wrote in your journal, etc). Skipped a day? Your streak counter resets.

    While tons of apps use streaks, Duolingo adds one twist that, as far as I know, is unique: The streak freeze. As you use the app, you earn the ability to buy streak freezes. Each streak freeze protects your streak for one full day of inactivity. Imagine: You’ve been studying Spanish dutifully every morning before breakfast for a month. But then one day you wake up feeling a little sick, sleep in a bit to recover… and since your routine was disrupted, you forget to practice Spanish that day. Most apps will say that you broke your 30-day streak, and the streak counter will reset the next time you practice. With Duolingo, though, if you had a streak freeze active for your account, your sick day would use up that streak freeze but your streak continues.

    Streak freezes dramatically increase the length of the streak you can build. Suppose you’ve got a 99% success rate at remembering to practice on Duolingo each day. Without streak freezes, you could expect your streaks to average around 100 days before they get broken. Impressive, yes! However, if you keep your account equipped with two streak freezes, you have to miss three days in a row to break your streak. With just a little bit of care, you can keep that streak going indefinitely. (If you didn’t take care and let chance dictate your streak length: that same 1% chance of forgetting gave you 100-day streaks in a world with no streak freezes. With streak freezes, left entirely to chance, you could expect your streak to last almost 30 years.)

    Longer streak lengths tap into two motivation centers in learner’s brains.

    1. Loss aversion It just hurts so much to lose something you “own.” If you have a long streak you’ll want to keep it. Each day your streak gets longer, your brain realizes it gets harder to replace if it breaks… so you care that much more about keeping it going.
    2. Identity At some point, after practicing a language and caring for a long streak, it stops being something you do and starts being part of who you are. “I’m a person who practices languages at least a little every day.” As Angela Duckworth writes in Grit, once you make an activity part of your sense of identity it makes it much easier to stick with it because your brain stops doing cost-benefit calculations.

    I’m not surprised that so many apps try to use streaks as a motivational tool — it’s a simple concept that’s simple to implement in almost any program. Streak freezes, on the other hand, require much more design and programming work.