mattepp's Beeminder/Anki experiment journal

Life
1

What

This is a personal beeminder journal thread specific to my experiments with Beeminding Anki, which begin today.

Why

I have used Anki seriously in the past, read just about everything at supermemo.com (although… evidently it’s been updated since then), and for years I’ve been interested in the potential of spaced repetition to improve my life in numerous ways. But I’ve never managed to sustain a daily habit of using Anki for more than 2 months, which entirely defeats the purpose. It is best used daily, indefinitely.

This is a perfect job for Beeminder! My problem is akrasia, pure and simple.

The only questions are what and how to track Anki study behaviors, and how to optimize my use of Beeminder to maximize results (habit reinforcement, memory retention) and minimize friction (spending time entering and managing data, avoiding Beeminder fees).

How

I’m beginning with this goal:

The fine print describes my intention and plans for how this will goal will work:

As of 2018-04-07, when I created this goal:

  • I have 4 decks (each for a separate knowledge realm).
  • I’m setting Beeminder to only require 2 points per week. This should be incredibly easy to accomplish
  • My intention is for this to be a trial goal. I will decide in two weeks (on 2018-04-21) whether to continue it. If I do, then I will trim safety buffer and increase the commitment to push for improvement.
  • I think that a good habit to build over a few weeks would be to clear at least 1 deck every day, and clear all decks at least 5 days/week. That would mean 22 points required each week.

For each deck, I must keep the “new cards per day” setting to at least 5. (That setting is currently at 10.)

This task makes my intention to review and tune in two weeks actionable. (That’s a Things URL Scheme link. They should call these “Things protocol links”, but whatever.)

I skimmed this forum thread on the Anki integration, and I’ve decided to put off any auto integrations for now. There are at least two kinds of value in manual data entry during a new goal experiment: Helping to reinforce the new habit, and having a more complete sense of my data to inform tuning my expectations, behaviors, and Beeminder goal settings.

Later, I may add one more more additional Beeminder goals for adding information to study into Anki. I’m not starting here, as this isn’t where I need help to get started. Adding notes to Anki is similar to making lists of todos for myself — something I’m great at. It’s the execution where I need help.

However, I think I may come to need the help of a Beeminder goal to add information to Anki. I imagine that as I successfully build the sustained habit of studying Anki cards, the novely of adding cards could wear off. Novelty is among my greatest motivators, so perhaps I’ll tend to neglect it. My intention with Anki is not to simply review information I’ve already added to date, but rather to continually add new information to my memory while retaining the old, on an ongoing basis. It would be possible for me to satisfy requirements of the goal I’m adding today, without fulfilling this intention. An additional goal requiring I add information for study should backstop my actual intention.

Feedback Welcome

If you’ve done something similar, I’d love to hear your stories and insights!

Constructive, helpful, direct feedback appreciated! I’m operating by Crocker’s Rules.

5 Likes
2

Super interested to see how this goes! Here’s a recent tweetstorm by an impressive person about how Anki changed their life and various smart-sounding tips for best using it:

1 Like
3

I used a lot of Anki when studying for my comprehensive exams a while back! I wrote a blog post about it which mentions the ever-increasing number of metrics I needed to beemind. It’s not a fully similar situation, since I had a deadline in mind, but I did use Anki regularly for three months.

3 Likes
4

Two additions to ANKI I have found to be useful:

Changing Goldilocks “Hard, Warm, Easy” to simply: did I recall it or not?

and

Updating the easing factor such that each card’s next due is set such that the projected recall success rate is (say) 85%.

I use this chaps code:

Important: You must not use an interval modifier in your deck options. In other words, your interval modifier must be set to 100% (no change) for all decks.

I’d suggest reading the related blog posts for the motivation behind it:

Related, regarding getting cards from the learning queue to review while maintaining the 85% recall accuracy:

And the steps I’ve copied:

All this hacking said, I wonder if just doing “vanilla” like Michael has done is the best approach. Maybe his mental horsepower overcomes ANKI’s inefficiencies! Though he does mention the Duolingo paper looking into more efficient methods.

The above hacks might be tweaks at the margin of effectiveness, and perhaps vearing into SRS-porn rather than just, you know, learning and retaining knowledge. I’ve only now begun using these hacks in earnest. I will report back here or elsewhere.

3 Likes
5

I’ve tried beeminding a few anki stats, and the one that by far worked the best for me was number of cards I’ve started studying and am caught up on - that is, not new, due, suspended, or buried. It encapsulates both adding new things and staying on top of reviews. And it allows some flexibility without sacrificing overall progress.

Like you, I started out entering data manually. I made it a little easier by creating a saved search that I can click and see the count and enter that, instead of having to do arithmetic every day.

1 Like
6

I agree the three buttons are unnecessary. The fact that it’s SRS at all is where the value comes from, not the precision of the algorithm. There are too many other factors in what you’ll remember and what you won’t that attempts to tweak the interval like this aren’t really relevant.

Similarly, I don’t worry about getting behind and catching up later - the biggest reason to make sure you don’t leave reviews undone is if you don’t have a system like beeminder to keep that from snowballing.

I do use the three buttons because they’re there though. I think of them as “guessed correctly” “remembered” and “just know”, and there generally feels like an obvious one to me.

3 Likes
7

So:

-is:suspended -is:new -is:due -is:buried -tag:marked

I like it- it’s easy to get carried away with making cards or futzing with superficial changes. Mature cards have to have gotten there somehow! In this case it’s just about maintaining pace.

I agree with the # Due being a number to track in beeminder is cross purposes. Due is a nudge for those not beeminding. In any case, I’d probably want to the number to be a percentage of some query rather than an absolute number.

My gut agrees. The “forgetting curve” is a mathematical model not a fact about memory. I’ve had a lot of success with ANKI’s factory algorithm in the past, and I’m going to just keep it vanilla. It’s decided! Thank you for weighing in.

2 Likes
8

Delete the “-is:marked” unless you’re using it some special way. Marks are just tags, they don’t affect reviews. (I mark and suspend cards when I notice that have issues that make them unstudyable, and mark without suspending cards that could use some minor cleanup but are studyable.)

I call this stat “maintained progress” which is somewhat descriptive but feels a bit clunky - any other ideas?

1 Like
9

Ah yes, sorry- that was not supposed to be part of the query as you specified. I use marks while reviewing and suspensions rarely for the same purposes- the card is poorly made and needs tending to. I usually do it on the spot, but if not I’ll mark it. The reason it’s part of my query is that I would not call a marked card (as I use them) worthy of a “point”. I think we agree.

I have named the query “Progress”- so we’re both pointing in the same direction.

I believe I will need an additional metric to nudge me to add new cards, though? While “new cards added” seems obvious, it again feels like a proxy. In this mode I’m reading sources and working on problems or exercises, and when I find a nugget I’d like to recall I always* put it in Anki.

* 50% of the time

1 Like
10

Looking forward to putting all these links in my head. Thanks for the homework! :wink:

1 Like
11

Super helpful, thanks for sharing! I also passed this on to my partner who’s in grad school. I find it fascinating to see learn about different ways of managing overwhelming study loads.

1 Like
12

Very elegant. I’ll give this a try if/when I get to the point where I need a push to add more notes.

— Or perhaps sooner, as I’m starting think that tracking “decks with zero reviews due” is a bit too simplistic. It’s rather lumpy: I can spend many minutes reviewing a deck and not get any points unless I actually complete all reviews. Four days in, this is beginning to feel like a problem, because I need to carve out long blocks of time to complete reviews so they “count”. It’s a disincentive to make incremental progress by reviewing as many cards as I have time during in-between moments.

Well-put. This is exactly how I think about the buttons. I find them to add little-to-no friction. In fact, imagining nly two buttons, I believe it would slow me down regularly with chewing over how to split hairs.

How about “Caught up”?

1 Like
13

After you catch up on reviews, you continue to increase this number by adding new cards and starting to study them. If you add more than necessary, you get some leeway to fall behind on reviews until when life gets in the way, until the combination of things coming due and the goal requiring more catches up with you.

In general the should be set to the amount of new cards that you want to add per day on average, though if you have a big backlog at the moment you might temporarily set it higher until that’s smaller.

2 Likes
14

I’m loving this idea, but I’m curious - is this an odometer goal? I have that search (minus marked) saved in Anki, but I’m confused on the best way to set the goal up in Beeminder.

2 Likes
15

I’m pretty happy with the habit I’ve made of clearing at least one deck per day, but it’s a struggle to find time for more than that. I’m dropping new cards/day setting down from 10 to 5 in hopes that this makes it more realistic.

1 Like
16

I’m also interested in how this can be made something you can Beemind!

ANKI has a statistics page, and there are addins that clear some of this up. From GitHub - matthayes/anki_progress_stats: Anki plugin that adds progress stats for learned and matured cards

Since one of the goals is to learn cards and have them eventually mature, this plugin tracks statistics for that specifically.

Using this addin, cumulative Net Matured Cards or Learned Cards might be a good odometer goal.

The number of cards I add per day varies so much (due to size- could be as short as a definition or maths proofs that take me some time) that

will have to be quite low to account for the longer cards, which is a shame but also perhaps a relief!

I’m going to stick with the above cumulative Net Matured Cards or Learned Cards as an odometer for now and will see how that goes.

17

Yes I forgot to mention that it’s an odometer goal, so you can just enter the number of cards that shows at the top of the window when you select the search.

1 Like
18

Remember this will also be limited by adding new cards, the difference is just that it won’t require you to continue to review them.

19

I’m confused as to how your query overcomes this. You could use your query as a do more and it will make sure you don’t “run down to zero (or whatever your number per day is)” i.e. cards in learning or mature but you’re not adding anything new. I assume that’s how you would do it?

Instead of using the odometer you could use do more with the above Net Mature or Learned and it’s exactly the same thing, right?

And you have the same problem: what is the right amount to do more?

It sounds like you have a clearer understanding and more experience. I wonder if you’d be so kind as to be specific as to how you use your query in Beeminder? What type of goal? Numerically how does it look? What kind of safety buffer? I hope that’s not asking too much.

1 Like
20

(Note: I have simplified irrelevant details about my own goals)

My anki query is ““deck:日本語” -is:suspended -is:new -is:due -is:buried”. I enter this number into an odometer goal. This means “cards in my Japanese I have started studying and am caught up on reviews for”.

I currently have my Japanese SRS beeminder goal set to 3 per day. This is not a very fast rate. I don’t have much time for study currently.

I find that if I’m adding cards at a steady rate and keeping up with reviews, I have 10 to 15 times as many reviews as new cards. So 10 new cards per day leads to needing to do 100 to 150 reviews per day. You should be able to use that combined with how much card creation and reviews you want to do in a day to pick a reasonable number. I largely found mine by trial and error; what makes me not decide to give up.

I haven’t looked into the net learned thing, but an advantage of an odometer over a do more goal is that if I miss entering things into beeminder for a day, I’ll still have the correct number when I do (I use anki on my phone and checking the search count requires my computer which I don’t always spend time at).

Safety buffers are not as meaningful for goals that are affected by anki cards coming due as they are for most things. Beeminder assumes that the number will stay the same if you don’t do anything, but in this case it will drop as cards become due if you don’t study them. I don’t tend to worry about the precise buffer amount much.

In practice I have cycles like this:

  1. Get excited and add items at a higher rate for a while, get a huge buffer.
  2. Burn out and stop doing any reviews for a while, things come due and my number drops quickly
  3. Beeminder catches up with me and makes me whittle down my backlog at the goal rate which is easy because it’s set at a reasonable card adding rate
  4. Usually after some time of this I get excited again and go back to step 1. Sometimes I don’t and I eventually catch up on all my reviews, whittling down my backlog at the daily rate of the beeminder goal, in which case I add that number of cards per day and keep up with reviews, until eventually I return to step 1.

Normally this kind of irregular studying would be terrible, but beeminder means that overall I do make progress, adding 3 items per day and keeping up with their reviews. As opposed to the situation where some people without beeminder get stuck on step 2 forever. It is less efficient than more regular review rates, but still much more efficient than studying without some form of SRS.

Another detail of my approach is that I have anki set to show new cards after reviews, so that I need to catch up on reviews in order to get new cards, to avoid the situation of adding new cards and not keeping up with the old ones. I do allow choosing to study one of my half dozen sub-decks of different types of cards though, so for example if I’m currently into grammar sentences I only need to catch up on my existing grammar sentences to get new ones, not my kanji etc. as well.

4 Likes