Beeminder Forum

How does a goal with a rate of 1 have +2 due?

The goal shows +2 due on Tuesday:

The road during that time only has rates of 0 or 1:

gym-road

How does this happen? Is it a bug?

2 Likes

I think it does count as a bug! The actual amount due is 1.0000000033333407 which is being conservatively rounded to 2. This is probably another argument for using a BigDecimal data type.

(Parenthetical discussion: what’s the severity of this bug? Overall everything is still quasi-impeccably correct, within like a billionth of a goal unit. Beeminder isn’t making you do an extra +1, just making you do it sooner than you ought to have to. But that’s a non-rhetorical question about severity. Sometimes that could be pretty unreasonable! And this part is parenthetical because even if the answer is “not very severe” it’s still very gross and I’d like to fix it.)

3 Likes

Thank you for the explanation!

Yeah, this really depends on the goal. I’m pretty sure I’ve had some goals where inputting a 1 takes so much work that the amount of work I’d have to do to enter 2 in a single day would be unmanageable.

Of course, if I derailed due to this bug I’d definitely call non-legit, but it might not be obvious in all circumstances and to all users that their derail was due to this bug, especially if their rate isn’t precisely 1.

2 Likes

I really feel the need to be pedantic again here, because you risk complicating your code and api while still having the same kinds of bugs.

You need a rational datatype, not a BigDecimal. Since you allow data to be input in hour format, you allow the user to introduce rational datapoints. Most of these cannot be represented with a BigDecimal.

20 minutes is 1/3 of an hour. You can’t represent that with a big decimal. Ironically, 1 / 3, added three times, is correctly rounded to 1. So 20 minutes “works” with floats as currently implemented, but not with a BigDecimal datatype, which would not do the rounding and just leave 10^(-precision) remaining, or in “exact” mode trying to add a 00:20:00 datapoint would throw an exception (because you can’t represent it exactly).

If I remember correctly, beeminder runs on ruby on rails, right? Looks to me like ruby has a rational data type out of the box. https://en.wikipedia.org/wiki/Rational_data_type#Ruby

5 Likes

Very severe - the amount you have to do is off by 100% and this should never happen.

Except it’s not, because that billionth gets rounded up to 2 instead of 1 which is huge for integery goals.

3 Likes

@eugeniobruno, thank you! That was helpful and edifying!

4 Likes

Gotta agree with zedmango here.

My “go to work” goal would be laughable ‘doing it earlier’. My “don’t do x” goal is physically impossible. I cannot not do the thing twice a day.

3 Likes

Yeah, other examples where it wouldn’t make sense include

  • take my daily medication
  • go to bed by a certain time
  • turn off my phone and lock it up for the night
  • choose a daily task for the next day
  • finish the task I chose on the previous day

A lot of my goals have been binary in this matter, and being told I need to do 2 of them in a day means the goal setup is broken.

7 Likes

Yep, I have the same kind of goals.

3 Likes

Addendum re rational vs bigdecimal: every float is a rational number. You could basically convert your whole database losslessly to them.

Possible scenario:
When you switch, you’ll write ra new rounding function that takes the dates of the datapoints. Sums that include post-switch datapoints are not rounded but kept in exact rationals. Sums that only include pre-switch datapoints are (again, losslessly) turned into floats and rounded according to the old float rounding.

Pareto something, right? :slight_smile:

4 Likes

I think this can be summarized as “Every time you store a float in a database instead of two integers you have made a mistake”

3 Likes

I definitely wouldn’t go that far. If it doesn’t need to be exact, eg. gps coordinates, automatic attenuation in dB for an audio player, position in a world for a mmo (except maybe EVE, where naively using floats would not work), floats are great.

CPUs can just crunch floats directly.

If the user can only input decimal numbers, BigDecimal is way better than floats, but also way better than rational. In general, working with BigDecimals will require a bunch more cycles per operations, but definitely not as much as Rationals.

Rationals are only needed here because beeminder allows things to be denominated in hours and part of them, which introduces the need for rationals. In general, Rationals will complicate things and cost more than BigDecimals computationally. Floats and big decimals allow you to do fairly straightforward SQL queries, whereas Rationals don’t, for instance (unless I’m mistaken).

Whenever I deal with time in my personal projects, I always use a milliseconds for durations and offsets from the unix epoch.

If beeminder internally used seconds or milliseconds for data input in hh:mm:ss format, BigDecimals would suffice.

4 Likes

De facto, floats cannot be compared for equality, even though programming languages mostly do allow you to use the == operator syntax on them. If you find yourself trying to compare the value of two floats for equality, you’re doing something wrong. The same applies also in SQL: should you try to execute a query like SELECT * FROM goals WHERE amount_due = 1.0 you would not find @narthur’s goal which is the topic of this thread, whose amount_due value is actually 1.0000000033333407.

For things like your examples of GPS coordinates and the like you never want to compare exact equality (perhaps approximate equality to within some epsilon, but never a query like the example above), so storing floats for that is fine.

If you want to do that kind of SQL query, then you have to use decimals or rationals. (The pg_rational extension for Postgres has a very good implementation of rationals for use in the DB.) Both work for this use-case, but floats don’t. So I’d flip your statement around: Rationals and big decimals allow you to do fairly straightforward SQL queries, whereas floats don’t in many cases.

But for many cases, the best way to represent fractional numbers is neither floats, nor decimals, nor even rationals stored as an (int, int) pair. Rather, it is rationals stored in the form of indexes into the Stern–Brocot tree. (The pg_rational extension I linked above actually uses this representation internally.) This post gives a good explanation of the subject.

5 Likes

And now the day has arrived:

I guess I’ll call the derail non-legit…

1 Like

Ok so I’m really confused about this situation. Is Beeminder now storing 1 as 1.0000000033333407 in general, leading to the situation where 1 tests as greater than 1? If so, seems like every goal would be non-legit derailing.

How did this goal end up like that? In what situations does 1 get stored as 1.0000000033333407, vs, say, 0.9999999966666593?

1 Like

It’s not that it’s storing every 1 as 1.0000000033333407; it’s that it’s storing a lot of (exact) 1s, but after a while summing them you get a sum with inexact rounding.

(Or something along these lines - not necessarily exactly this)

2 Likes

I don’t think beeminder ever queries for datapoints exactly equal to something. It either does sums or asks for <= or stuff like that.

1 Like

Wow - so this is going to happen to every integer goal at some point!

That makes this bug a lot worse than I thought - integer goals no longer work correctly. That’s a huge loss of functionality.

Is this a result of the recent rounding/display precision change to Beeminder?

1 Like

Wait, before sounding the alarm - my knowledge of floating point/numerical representations of numbers in computer science is very, very shallow. I tried summing exact ones for a while:

>>> for i in range(10000000):
...   if k != float(i):
...     print(k, float(i))
...   k += float(1)

and the sum are exact, at least up to 10 million summed ones. Maybe if you sum stuff like 0.2 + 0.9 + 0.3 + 0.5 + 0.15 + 0.35, eventually you get the rounding error.

Maybe we should ask @zzq, it seems like they know what they’re talking about :smiley: (I had no idea about the existence of the Stern-Brocot tree…)

3 Likes

This is a really good point. Using seconds internally would allow a precision of 1/3600 unit for non-time goals, which should be sufficient. And it would allow for most fractions to be stored perfectly, since 3600 has so many divisors.

The only catch is that 7 might be important, since there are 7 days in a week. A goal with a weekly rate of 1 needs 1/7 unit each day.

So ideally Beeminder should store everything in sevenths of a second - that is, units of 1/25200.