Beeminder's Turquoise Swath?

Tech
#1

What sort of fit does Beeminder perform when you enable the “Turquoise Swath”?

I’m wondering because I noticed wild variations sometimes appearing before or after the existing data points on various graphs that have it enabled. Have the bees considered using Gaussian processes to perform the fitting a la the Automatic Statistician? Depending on the kernel selected, you can provide a much stronger prior on what curves should look like, reducing those wild swings outside the existing data.

There has even been some work on compositional kernel construction (e.g. this paper, which might be useful for capturing structure like a predictable flurry of activity at the end of the month or breaks on the weekend.

I’m not entirely sure what the computational demands are for fitting something like a Gaussian process compared to what Beeminder does now but just wanted to throw the idea out there in case it sparked ideas for features down the road.

3 Likes
#2

It’s a polynomial fit! Here’s the very, very complete answer: https://github.com/beeminder/road/blob/master/src/polyfit.js [1]

Thanks for the pointer to something potentially much better! I have a bunch of notes collected on that question as well…

[1] That’s right, all the code that generates the graphs is open source now! Well, as soon as we finish the migration to the new Beebrain: [oops, I originally linked to an internal thread here]

3 Likes
#3

Thanks - this is great!

It looks like the fit is always a cubic - is that right? Is there any particular reason why we should expect beeminder trajectories to be cubic?

Just curious, but what sorts of ideas do you have for improving the Turquoise swath?

2 Likes
#4

Yeah, there’s no good justification for that choice of polyfit!

Here’s our list of ideas/notes:

  1. Ridge regression, scipy.optimize.fmin_l_bfgs_b
  2. http://en.wikipedia.org/wiki/Tikhonov_regularization
  3. http://en.wikipedia.org/wiki/Kriging
  4. Butterworth filters?
  5. https://en.wikipedia.org/wiki/Kernel_density_estimation
  6. R’s geom_smooth with Loess smoother
3 Likes
#5

Cool!

Perhaps I’m using it incorrectly, but my primary use case for the swath is to less to smooth out existing data and more to make sane predictions. Whatever improves that is a win in my book :grin:.

One other option to consider that’s somewhat similar to Kriging, is some sort of Bayesian regression in which you perform inference over both the type of curve (sinusoidal, linear, parabolic, cubic,…) and the specific parameters of the curve. Depending on your setup, you can tradeoff between the complexity of the curve and fit in a principled way.

1 Like
#6

I may be completely wrong here @dreev, but I think kriging is a special version of Gaussian Process.

1 Like
#7

@mufflon - that’s my understanding, too. It looks like kriging originally came out of geostatistics.

I’ll link a couple other resources that were useful for me when first learning about Gaussian Processes more generally:

2 Likes