Beeminder Forum

Progress Report on Yellow Brick Half-Plane

Latest status: Bee is deep in refactoring of how the “no mercy” setting works so that we can generalize that so you can choose how much safety buffer you get when you derail which turns out to be an important feature for killing the last of the auto-widening yellow brick roads, which is just about the last in a long list of prerequisites for killing the concept of lanes. Phew! We’re getting there!

Here’s how @bee put it in a daily beemail:

Do you have value for being able to parameterize how much mercy you get when you derail a goal? I think mostly the feedback we’ve heard about the no-mercy option is that people want less mercy and are frustrated that our idea of “no mercy” still gives them 2 days off because they want for everything to always be an eep day no matter what. However, because of technical reasons, you can’t have an eep day immediately after a derail, so… well, what I’m talking about here does not fix that want. (Not that the technical reasons are insurmountable; just not on the table quite yet because, well, it’s technical.)

But does anyone ever wish they could have like 3 days off after a derail, or 2 weeks, or something?

Here’s the deal: currently with new-world-order weight roads (i.e. razor roads), if you have an up day and derail you’ll be recommitted right on the razor’s edge.

I love that because I hated how in the old version if I derailed my weight goal, my road moved to put my offending datapoint right in the middle of the road, with a big wide buffer above me. That meant that one bad day not only cost me money, it was then allowed to compound into many bad days in a row, which made it really easy to backslide. Now if I screw up, the road recommits with me right on the edge – I get a little buffer above where the road previously was (because my derail point is necessarily above the old road), but I still have to stay in the game and keep making progress.

A better implementation would let you adjust how much room you get on a recommit by having a parameterized “mercy” setting.

What I’m imagining is that this would be quite a bit like specifying your auto-ratchet amount. For do-more goals, you’d specify a number of days. For weight loss roads, and do-less roads too, you’d want to specify how much mercy you got on recommit in terms of units of buffer. If you liked the old style behavior with weight roads where you got a bunch of extra buffer from derailing, you could set your amount-of-mercy equal to your maximum daily fluctuation. If you wanted less buffer, you could decrease that amount, anywhere down to ‘0’, which would be like the current (new) behavior.

Anyhow, this is just kind of spitballing, except I’m probably going to do it.

5 Likes

We obviously got super distracted by a million things – many of which are very related to YBHP, like Javascript graphs! We actually planned for YBHP to be a prereq for Javascript graphs but then @saranli, in his amazingness, managed to port all the hideous lanes cruft, fully backward-compatibly, so now we have a single new codebase for all of that from which to proceed.

Oh and we did ship Generalized Mercy – https://blog.beeminder.com/mercy/ – and forgot to update this thread [sheepish].

The remaining prereq is finishing death-to-autowidening-roads. All new roads have been sans autowidening for a quite a while and we’ve just been putting off some painful migrations for existing roads. (Upside is that the longer we wait the fewer there are to migrate and the less daunting that job is. :slight_smile:) Plus other loose ends we’re not happy with in the new “razor roads” for weight loss.

Anyway, it’s embarrassing to be “so close” on this this long but it remains true and remains important!

1 Like

Thank you for this! Very exciting that we’re so close.

Would you please clarify how the width of the new, sans-autowidening roads is determined? Is it just 1 day’s worth?

And would you clarify the relationship between the width of the road, the color of the goal, and the number of days till derailment? Is it just that the lanes correspond to different colors and the colors represent different numbers of days till derailment?

The idea is to use a zero-width road but shift it to give you enough safety buffer. This stuff really wants a blog post or other documentation but has to be gleaned in tweet-sized pieces for now:

http://beeminder.com/changelog#2743

We do like the colors a lot and want those to stay, just no actual lanes. So the yellow brick road has no width but if you’re on the wrong side of it you’re in the red (will derail within 24 hours), if you have 24-48 hours (< 2 days) till you derail then you’re in the orange zone approaching the YBR, a little further away is the blue zone (2 days of buffer), and then the green zone (3 or more days of buffer).

So, yeah, with YBHP the colors encode nothing more nor less than amount of safety buffer.

2 Likes

No, sorry, I wasn’t clear. I meant now, not after YBHP.

You said:

I’m asking about the roads now, which are sans autowidening.

How is the width of roads now determined? Is it just one day’s worth?

And do the lanes now correspond to different colors and different numbers of days till derailment?

I’m confused. Shouldn’t derailment happen as soon as you cross the wrong side? Or if you’re on the wrong side at the next deadline? (I’m now asking about after YBHP.)

What’s the new derailment condition?

You can be in the red during the current day, before your deadline. In fact, on an emergency day, you start out in the red. If you haven’t moved back to safety (any other color) by your deadline (default: midnight), you derail.

I very, very strongly suppose this will not change.

And I think @dreev’s answer is also true now:

@howtodowtle no - unfortunately it’s a lot more complicated than that currently. Right now, you derail if, at any time, you’re in the red and you were in the red yesterday.

I’m asking @dreev what the new derailment condition under YBHP will be.

Isn’t that (the current situation) de facto what I wrote, just in other words?

If you start in the red today and were in the red yesterday, you derail. That’s the same as being in the red and failing to get back on the road by midnight.

It’s a lot more complicated than that. Check out:

1 Like

@dreev sorry - I was confused.

Wait a second - autowidening was only for weight loss?

https://blog.beeminder.com/roadwidth/

So that’s just for weight loss goals, right?

I still don’t understand this.

There’s at least three ways the road width can change:

  1. You use the API to set lanewidth on the goal.
  2. You change the slope of your road (via road dial or derail or any other means), which automatically changes lanewidth as if by #1.
  3. The goal is a weight loss goal and you would be in the red today without being in the orange yesterday.

Type #3 is called “autowidening”. Type #2 (which applies to all goals) is automatic but it doesn’t have a name that I’m aware of.

1 Like

As I understand it, #3 is no longer true:

https://blog.beeminder.com/roadwidth/

Some experimenting with changing road width shows that the colors now are inconsistent but generally go with the lanes, not the number of days till derailment, sadly.

Relevant:

So basically beeminder will hit maturity when YBHP is here and all these weird bugs go away.

@dreev With YBHP will derailments occur only if you’re on the wrong side at your deadline? So that’s why it’s “within 24 hours”?

Correct, except we’re not done migrating existing roads and have other loose ends before we’re happy with declaring auto-widening fully dead.

Right. Gory details: Mother of Bugs: Two Consecutive Days In The Red

1 Like

@dreev thanks!

Any chance we’ll get color coding based on the “lead days” setting, like I suggested here?

So rather than “red” triggering the day before, it would trigger at a number of days based on the “lead days” setting.

Status report on the key visual aspect of the Yellow Brick Half-Plane project, thanks to Uluc (@saranli)! This is all deep in the guts of Beebrain and, as usual in this thread, users don’t need to care about this at all, other than how the graph ends up looking, which is very much in flux.

Our technical term for this is days-to-derail (dtd) isolines. Oh and the yellow brick road (ybr) itself is now assumed to be zero width. It’s the bright line that you derail if you cross. Well, if you end the day on the wrong side of. So the ybr is the 1-day isoline. (Technically 0-day in the implementation right now. We have a big off-by-one error in everything that follows. See item #0 below.) The 2-day isoline is the boundary between being in the orange and being in the blue. Beyond 3 days of safety buffer is green. Seems simple enough so far but this is much easier said than done and Uluc is a literal genius.

(My favorite quote from when we were hashing this out and he wrote a Matlab script to figure out how it should work: “I just lifted the PPR function to the continuous domain as a vector, taken the integral of its negation and plotted a curve through its isopoints.”)

Here are questions and comments for each other as we get ready to ship this for a few intrepid guinea pigs:

  1. We’re currently drawing this with the first region closest to the road as red but you’re actually safe for more than 24 hours in that region so we need to shift all the colors. It’s the bad half-plane that’s red. The shading for the oinkzone (nee pinkzone) is kinda perfect but then we need another way to visually represent the oinkzone. Grayed out? Something like caution tape?

  2. The ppr() function in broad.js enforced ppr=0 for t < asof. We put in an option to that function to force computation of past PPRs as well so we can show colored regions for t < asof. Otherwise the entire region before asof appeared green, which is technically correct but not very useful. We want to see what the dtd margins looked like in the past.

  3. We haven’t yet implemented the yellow guiding lines for dtd > 7, but that will be really easy. We have the green region up until dtd - 7 for now, but any region between any two dtd values can be drawn. The algorithm Uluc devised can generically compute isolines for any dtd value, which are by definition guidelines anyway.

  4. The current color selection is too saturated; region colors should probably be less pronounced.

  5. If you open roadeditor_test.html in the browser, you should be able to edit the road and see the regions change in real-time.

  6. Regions can be somewhat unintuitive for doless goals, but we’re fairly certain they’re correct. Inflection points in isolines either coincide with t=t_node for inflection points on the road, or when they cross boundaries across which the ppr function changes.

  7. Palette optimization has not yet been done for png outputs, but we can do that once and for all once we decide on the colors.

  8. Uluc is lobbying for not cutting of the green region at the 7-day boundary as Dreev suggested. It might be ugly or confusing. Maybe much lighter colors but having the green region cover the half-plane all the way out. We can still have the yellow guiding line to indicate the special 7-day boundary.

  9. Still need to change the color and style of the centerline-cum-bright-derailment-line. Dreev was thinking it could be bright red or something. (The dotted line was a cute thing for emulating the painted line in the center of a physical road and doesn’t make sense in the new world order.)

Finally, there are some technically incorrect aspects of the isolines for do-more goals with downward sloping segments that we’re working on. Some parts of the isolines are closer to the actual derailment points on the road since they are offset from further parts of the road. A picture should make that make more sense:

Normally, that should just be a flatline up until the point where the isoline goes above the inflection point. (Uluc was hoping to avoid another pass over the isolines for monotonicity enforcement but probably not actually a big deal if that’s the best way to do it.)

Here is a do-less example:

For do-less goals with upward vertical slopes on the road, Uluc ensured consistency with very steep segments by shifting the dtd isoline by 2*delta where delta is the vertical displacement associated with that segment. That’s why you see weird looking blocky parts on the do-less graph above.

To get a better understanding, compare the following two graphs:

This way, regions don’t jump around when going from near-vertical to true vertical. We should probably do something similarly consistent for flat road segments, says Uluc. One possible interpretation for this vertical fix is that the PPR would be twice as much as the vertical displacement on the road. The ppr() function may also want to reflect that for consistency, but that’s probably an extreme edge case.

(PS: the centerline of the road appears wrong in those screenshots because it’s from the dynamic road editor which shows the centerline pre-edits.)

3 Likes

what is asof?

I’ll put in my vote in agreement with Uluc on point 7.

2 Likes

me too!