What's the exact algorithm for Beeminder saying you've derailed?

Bugabee
1

Intuitively it seems like it would be “the clock strikes midnight (or whatever your deadline is) and your datapoint is still in the red”.

But it’s not exactly that, and it’s hard to make it be exactly that because it’s hard to guarantee that a given piece of code runs at a certain exact time.

So what it actually is is “yesterday’s datapoint is red”. If at any time Beeminder notices that that is true then you instantly derail. Normally that happens within seconds or at most minutes after your actual deadline (when it’s officially the new day) and everything is great.

But it can also happen because you, say, edited yesterday’s datapoint, and that’s surprising. Maybe still correct, but surprising.

​If I wanted to defend this I’d cite worse-is-better – see the parenthetical towards the end of https://blog.beeminder.com/magic/ – and explain how nice and simple that makes the implementation. Namely, when the deadline hits it becomes officially the new day and we have a dirt-simple check, “red yesterday”, that doesn’t depend on when the check actually happens which in fact can be arbitrarily far after the deadline in case of the queues being backed up or network delays or god-knows-what.
​
​That may not be a good apologia though! “Derails happen only when you hit the deadline while in the red” may be The Right Thing. But how to get there from here? We could get a little closer by defining a window: if it’s not within X minutes of the deadline then let a red datapoint yesterday slide. But that’s ugly and fragile and confusing and magical.
​
​There’s probably a way to have the best of all worlds but we haven’t thought of it yet…

1 Like
2

I think this is the best of all worlds, because if you discover you entered the wrong data and need to correct it, you can fix the problem and you’ll be charged correctly.

1 Like
3

Here’s a specific use case where it comes in handy: I have beeminder goals for various parts of my bedtime routine. One of them is to lock my phone up in my commitment-contract time safe to help me sleep.

Now with my phone locked up, I can’t enter data on any goals. I could use my computer, but that’s not what I want to do before bed. I could have the phone out and lock it up last, but then the phone is out distracting me.

But this way, I can pre-enter the data, write in my daily log notebook, cross it off when I do it, and if there’s something I didn’t do, delete it in the morning and I will still be charged. The paper notebook is a reminder to make sure I delete any pre-entered data that isn’t accurate.

(of course, now that I’m writing all this out, I realize that in some sense the “correct” way to go would be write it in my notebook and then make the goal “did I do it yesterday?” Maybe I’ll switch to that system…)

1 Like
4

From a programming perspective, I agree that you want something that’s robust, as opposed to ugly and fragile and magical. I’d implement it something like this:

  1. Add a database field to the goal object named something like last_derail_check, which stores a (nullable) timestamp.
  2. Implement something like this pseudocode as your check logic:
if yesterday was red:
  if last_derail_check is null || last_derail_check < yesterday's deadline:
    mark goal as derailed

last_derail_check = current timestamp

i.e. keep track of the last time you checked for a derailment, and if you’ve already checked today (since the deadline), avoid derailing. (Also, you may want to nullify the last_derail_check field if the user edits the deadline to be earlier, and possibly some other edge cases like that.)

You can run this check at any time, even repeatedly, and it’s idempotent: running it twice in a row is logically identical to running it just once. You can safely replace the current derailment check with something like this, and it gives you the best of both worlds: it’s no more fragile, and it doesn’t derail in the middle of the day. (That is, only the first time in a given day that this check runs for a given goal is there a chance for derailment.)

2 Likes
5

Ah, smart! I’m liking this. Thank you for helping us think this through! Needs careful thought about edge cases, as you say. But it’s sounding right. If you edit past data and make yesterday red, that will (I think it’s safe to assume, though this is another edge case to carefully walk through) be after the first derail check that happened right after yesterday’s deadline. So it doesn’t matter what color any datapoints become. At today’s deadline, as long as you eke back onto the road before it hits, the first check after the deadline will not be red-yesterday.

1 Like