Beeminder Forum

Closing the do-less loophole

As promised, now that we’ve closed the infamous, enigmatic do-less loophole I can tell you what it was! Or, to further pique your curiosity, or in case you’re also curious how we managed the transition, I can first show you what we emailed to everyone with a do-less goal last week:

If you don’t know what we mean by the do-less loophole, consider yourself lucky and ignore this email! If you do then you know that it partially ruins do-less goals or at least makes them stupidly complicated. Fortunately the long national nightmare is at an end. If you’ve been abusing that loophole, be warned! In one week (Aug 12) it will stop working. Ending the day in the red means derailing, no caveats. We recommend pretending that’s the case starting right now so you can make sure you’re used to the more elegant, fair, and harsher Beeminder in time. (Of course if you forget and derail because of this, just talk to us. And any general concerns, definitely reply to this email.)

Thanks for beeminding!
The Bee Team

Ok, are you ready to be retrospectively horrified? I actually explained this as early as 2015 in a footnote which I shall now self-plagiarize. (@drtall also has an explanation with nice concrete examples.)

What the Heck was the Actual Loophole?

The criterion for derailment used to be “in the red yesterday and also today”. Never mind why but there were Reasons, and for most goal types it worked fine. But for Do Less goals it meant that if you exceeded the yellow brick road by less than the daily rate then at midnight (or whatever your deadline) when it checked it said “you were in the red at the end of yesterday but now today you’re orange, so you’re ok”. This didn’t totally ruin the point of the Do Less goal. You still had to maintain the average rate you committed to. But in the akratic extreme it meant you could do twice the daily rate every other day (and nothing in between). So if you were right on the top edge yesterday then the next morning you’d start right at the centerline. That meant you could do one day’s worth to hit the top edge and another day’s worth into the red. The next day you’d flatline back to the very top edge of the road again (where doing anything >0 would put you in the red and derail you since you ended the previous day in the red) and the cycle could repeat.

In short, you could sometimes exceed the nominal limit by a small amount and then eke back into safety as the day rolled over.

That was ridiculous and no reasonable person wanted that even though plenty of users (me included) got used to it and abused the crap out of it at various times.

Into the Weeds

Fixing that had a chain of dependencies. We had to change what happened when you derailed, having the yellow brick road jump to where you ended the beemergency day, not the day after. (UVI #3083) That also fixed a lot of problems with beemergencies after derailing (UVI #3084) and having true zero-mercy derails (UVI 3092), which meant before any of this we had to generalize the concept of no-mercy derails (UVI #2734). Also the new rerail criterion meant past derailments would no longer be visible on the graph as red datapoints so we needed to first add a separate visual indicator of past derailments (UVI #2760, UVI #2761, UVI #2768, etc).

Here’s discussion of some of the other big problems besides the do-less loophole that were fixed by getting rid of “today and yesterday both red” as the derailment criterion:


And here’s my sketch of the old and new derail/rerail behavior that’s behind all this:

In conclusion, code is hard but we are very clever and did all the things.

(Btw, Yellow Brick Half-Plane is a little like the above in some ways but 10 times hairier. The deceptively simple-sounding bug to fix in that case is “don’t allow extra leeway beyond exactly what you dialed the road to” or “make the colors always reflect the number of safe days”.)

2 Likes

I don’t understand the bottom row of the chart (new behavior).

Thank you so much for fixing the “two red days” bug!! I’ve been complaining about it for a while! I’m very happy and grateful for these changes. :heart: :heart_eyes: :honeybee: :honey_pot:

3 Likes

Funnily enough, I immediately thought of use cases for that bug as soon as I read the explanation :joy: Not that that’s a reason not to kill it, but it’s probably a good reason not to tell me about bugs with interesting effects on goal requirements…

2 Likes

Yeah, that’s confusing looking. It’s trying to show the road jumping to where the datapoint ended (plus extra safety buffer aka mercy) on the beemergency (eep) day. So top left is the do-less loophole happening, bottom left is do-less derailing/rerailing without the loophole. Top right is the old derail/rerail behavior for do-more and bottom right is the new behavior. Old vs new doesn’t matter much for do-more.

This is fascinating to me right now.

You caught me with a notebook full of diagrams just like those ones, and a walk to and from work today spent in part thinking about my own derail criterion.

I’d love to know: Why??

My own code is based on events (“rate changed”, “recommitted”) rather than, as I gather, solely on cumulative “road bricks”: perhaps the reason is connected with that?

For what it’s worth, my own recommit “rerail” is the day following the derail, and (tentatively as of this morning’s walk anyway) my derail condition is similar to: “did the data fall below the goal line at the end of the last ‘goal day’, and no derail was yet recorded for that day?” (there is no do-less yet in my system).

I certainly, by the way, don’t mean to say that using events rather than just cumulative points is superior in all respects: in fact I have demonstrated that very effectively by getting myself into a lot of trouble by means of not sufficiently carefully working through the consequences of my own model up front. I’m only just starting to dig myself out of that.

1 Like

It allowed a kind of stateless/timeless check for derailment. Beebrain’s test for derailment (Beebrain is the part of the code that takes the data and settings and outputs stats like amount of safety buffer, plus the actual graph images) was “yesterday and today both in the red” because that way when the deadline would hit and the goal was rebrained, Beebrain saw that what’s-now-yesterday was red and what’s-now-today was also red and concluded that the goal was derailed. Then we’d rerail the goal and today would no longer be red and Beebrain would see that everything was fine, without having to keep track of pesky things like exactly when you were asking.

Now that we’ve changed the rerail behavior so that the road jumps to the eep day datapoint, Beebrain only has to care about what’s-now-yesterday when it processes the goal when the deadline hits. If what’s-now-yesterday is red then you derailed. After rerailing, yesterday won’t be red anymore.

Thank you. Now that you pointed that out, it seems obvious :slight_smile: