mattepp's Beeminder/Anki experiment journal


#1

What

This is a personal beeminder journal thread specific to my experiments with Beeminding Anki, which begin today.

Why

I have used Anki seriously in the past, read just about everything at supermemo.com (although… evidently it’s been updated since then), and for years I’ve been interested in the potential of spaced repetition to improve my life in numerous ways. But I’ve never managed to sustain a daily habit of using Anki for more than 2 months, which entirely defeats the purpose. It is best used daily, indefinitely.

This is a perfect job for Beeminder! My problem is akrasia, pure and simple.

The only questions are what and how to track Anki study behaviors, and how to optimize my use of Beeminder to maximize results (habit reinforcement, memory retention) and minimize friction (spending time entering and managing data, avoiding Beeminder fees).

How

I’m beginning with this goal:

The fine print describes my intention and plans for how this will goal will work:

As of 2018-04-07, when I created this goal:

  • I have 4 decks (each for a separate knowledge realm).
  • I’m setting Beeminder to only require 2 points per week. This should be incredibly easy to accomplish
  • My intention is for this to be a trial goal. I will decide in two weeks (on 2018-04-21) whether to continue it. If I do, then I will trim safety buffer and increase the commitment to push for improvement.
  • I think that a good habit to build over a few weeks would be to clear at least 1 deck every day, and clear all decks at least 5 days/week. That would mean 22 points required each week.

For each deck, I must keep the “new cards per day” setting to at least 5. (That setting is currently at 10.)

This task makes my intention to review and tune in two weeks actionable. (That’s a Things URL Scheme link. They should call these “Things protocol links”, but whatever.)

I skimmed this forum thread on the Anki integration, and I’ve decided to put off any auto integrations for now. There are at least two kinds of value in manual data entry during a new goal experiment: Helping to reinforce the new habit, and having a more complete sense of my data to inform tuning my expectations, behaviors, and Beeminder goal settings.

Later, I may add one more more additional Beeminder goals for adding information to study into Anki. I’m not starting here, as this isn’t where I need help to get started. Adding notes to Anki is similar to making lists of todos for myself — something I’m great at. It’s the execution where I need help.

However, I think I may come to need the help of a Beeminder goal to add information to Anki. I imagine that as I successfully build the sustained habit of studying Anki cards, the novely of adding cards could wear off. Novelty is among my greatest motivators, so perhaps I’ll tend to neglect it. My intention with Anki is not to simply review information I’ve already added to date, but rather to continually add new information to my memory while retaining the old, on an ongoing basis. It would be possible for me to satisfy requirements of the goal I’m adding today, without fulfilling this intention. An additional goal requiring I add information for study should backstop my actual intention.

Feedback Welcome

If you’ve done something similar, I’d love to hear your stories and insights!

Constructive, helpful, direct feedback appreciated! I’m operating by Crocker’s Rules.


#2

Super interested to see how this goes! Here’s a recent tweetstorm by an impressive person about how Anki changed their life and various smart-sounding tips for best using it:


#3

I used a lot of Anki when studying for my comprehensive exams a while back! I wrote a blog post about it which mentions the ever-increasing number of metrics I needed to beemind. It’s not a fully similar situation, since I had a deadline in mind, but I did use Anki regularly for three months.


#4

Two additions to ANKI I have found to be useful:

Changing Goldilocks “Hard, Warm, Easy” to simply: did I recall it or not?

and

Updating the easing factor such that each card’s next due is set such that the projected recall success rate is (say) 85%.

I use this chaps code:

Important: You must not use an interval modifier in your deck options. In other words, your interval modifier must be set to 100% (no change) for all decks.

I’d suggest reading the related blog posts for the motivation behind it:

Related, regarding getting cards from the learning queue to review while maintaining the 85% recall accuracy:

And the steps I’ve copied:

All this hacking said, I wonder if just doing “vanilla” like Michael has done is the best approach. Maybe his mental horsepower overcomes ANKI’s inefficiencies! Though he does mention the Duolingo paper looking into more efficient methods.

The above hacks might be tweaks at the margin of effectiveness, and perhaps vearing into SRS-porn rather than just, you know, learning and retaining knowledge. I’ve only now begun using these hacks in earnest. I will report back here or elsewhere.


#5

I’ve tried beeminding a few anki stats, and the one that by far worked the best for me was number of cards I’ve started studying and am caught up on - that is, not new, due, suspended, or buried. It encapsulates both adding new things and staying on top of reviews. And it allows some flexibility without sacrificing overall progress.

Like you, I started out entering data manually. I made it a little easier by creating a saved search that I can click and see the count and enter that, instead of having to do arithmetic every day.


#6

I agree the three buttons are unnecessary. The fact that it’s SRS at all is where the value comes from, not the precision of the algorithm. There are too many other factors in what you’ll remember and what you won’t that attempts to tweak the interval like this aren’t really relevant.

Similarly, I don’t worry about getting behind and catching up later - the biggest reason to make sure you don’t leave reviews undone is if you don’t have a system like beeminder to keep that from snowballing.

I do use the three buttons because they’re there though. I think of them as “guessed correctly” “remembered” and “just know”, and there generally feels like an obvious one to me.


#7

So:

-is:suspended -is:new -is:due -is:buried -tag:marked

I like it- it’s easy to get carried away with making cards or futzing with superficial changes. Mature cards have to have gotten there somehow! In this case it’s just about maintaining pace.

I agree with the # Due being a number to track in beeminder is cross purposes. Due is a nudge for those not beeminding. In any case, I’d probably want to the number to be a percentage of some query rather than an absolute number.

My gut agrees. The “forgetting curve” is a mathematical model not a fact about memory. I’ve had a lot of success with ANKI’s factory algorithm in the past, and I’m going to just keep it vanilla. It’s decided! Thank you for weighing in.


#8

Delete the “-is:marked” unless you’re using it some special way. Marks are just tags, they don’t affect reviews. (I mark and suspend cards when I notice that have issues that make them unstudyable, and mark without suspending cards that could use some minor cleanup but are studyable.)

I call this stat “maintained progress” which is somewhat descriptive but feels a bit clunky - any other ideas?


#9

Ah yes, sorry- that was not supposed to be part of the query as you specified. I use marks while reviewing and suspensions rarely for the same purposes- the card is poorly made and needs tending to. I usually do it on the spot, but if not I’ll mark it. The reason it’s part of my query is that I would not call a marked card (as I use them) worthy of a “point”. I think we agree.

I have named the query “Progress”- so we’re both pointing in the same direction.

I believe I will need an additional metric to nudge me to add new cards, though? While “new cards added” seems obvious, it again feels like a proxy. In this mode I’m reading sources and working on problems or exercises, and when I find a nugget I’d like to recall I always* put it in Anki.

* 50% of the time


#10

Looking forward to putting all these links in my head. Thanks for the homework! :wink:


#11

Super helpful, thanks for sharing! I also passed this on to my partner who’s in grad school. I find it fascinating to see learn about different ways of managing overwhelming study loads.


#12

Very elegant. I’ll give this a try if/when I get to the point where I need a push to add more notes.

— Or perhaps sooner, as I’m starting think that tracking “decks with zero reviews due” is a bit too simplistic. It’s rather lumpy: I can spend many minutes reviewing a deck and not get any points unless I actually complete all reviews. Four days in, this is beginning to feel like a problem, because I need to carve out long blocks of time to complete reviews so they “count”. It’s a disincentive to make incremental progress by reviewing as many cards as I have time during in-between moments.

Well-put. This is exactly how I think about the buttons. I find them to add little-to-no friction. In fact, imagining nly two buttons, I believe it would slow me down regularly with chewing over how to split hairs.

How about “Caught up”?


#13

After you catch up on reviews, you continue to increase this number by adding new cards and starting to study them. If you add more than necessary, you get some leeway to fall behind on reviews until when life gets in the way, until the combination of things coming due and the goal requiring more catches up with you.

In general the should be set to the amount of new cards that you want to add per day on average, though if you have a big backlog at the moment you might temporarily set it higher until that’s smaller.


#14

I’m loving this idea, but I’m curious - is this an odometer goal? I have that search (minus marked) saved in Anki, but I’m confused on the best way to set the goal up in Beeminder.


#15

I’m pretty happy with the habit I’ve made of clearing at least one deck per day, but it’s a struggle to find time for more than that. I’m dropping new cards/day setting down from 10 to 5 in hopes that this makes it more realistic.