I have a pull-ups and push-ups goal, and as soon as I reach 8 days buffer, I will increase the rate. I have not yet been able to do this
Yeah, it’s hard to know until you try. I think it’s a good idea to only increase one goal, and only increase it by a little bit.
It’s also worth thinking about what you’re going to drop or give up in order to give yourself the time and energy to meet the increased rate. If you can’t identify something specific, there may not be room, or you may drop something important under the pressure.
More thoughts on this that started as thinking out loud in a daily beemail (prompted by @mary making an incisive counterpoint):
Derailing early and often until you reach your Motivation Point is great but then won’t you just continue at Peak Awesomeness without ever derailing again?
I think there are different ways it can play out. My personal favorite, and perhaps the one most compatible with this “everyone should derail as much as possible” thesis, is where you settle on an amount of money that motivates you during normal circumstances and when anything unusual happens you pay to derail and are happy about that.
It’s like a natural way to define the right amount of flexibility without having to worry about fine print (which I personally kind of dislike). The criterion is simply “are these circumstances extenuating enough that I’m ok paying this pledge?”
Of course there’s still a balancing act – you have to make sure you don’t akratically decide to derail too often.
The point is, there’s not a single Motivation Point. It depends on a million things and changes over time. But, so I claim, more derailing always correlates with Beeminder doing more motivating.
PS: And now I’m thinking about a follow-up essay about “Motivation Points vs Success Spirals” since @nick has a philosophy of beeminding that might be pretty incompatible with this whole derailing-is-not-failing thing…
It really depends on the goal. For some goals, you never want to derail, so you set the pledge really high. For others, you want the flexibility of being able to choose whether or not to derail and pay X, and you want to set X ahead of time to maximize awesomeness.
It’s not at all clear to me that the value of X that maximizes awesomeness is the same as the value of X that maximizes revenue.
Suppose 15% of the time I want to stay up late bad enough to pay $5, but not bad enough to pay $10, and 10% of the time I want to stay up late bad enough to pay $10.
At X=5, I pay $5 25% of the time, or an average of $1.25.
At X=10, I pay $10 10% of the time, or an average of $1.
Now suppose 5% of the time I want to stay up late bad enough to pay $5, but not bad enough to pay $10, and 20% of the time I want to stay up late bad enough to pay $10.
At X=5, I pay $5 25% of the time, or an average of $1.25.
At X=10, I pay $10 20% of the time, or an average of $2.
So in the first scenario, X=5 maximizes revenue, but in the second scenario, X=10 does.
Now, which maximizes awesomeness? Looking back from outside the akrasia horizon, only a small portion of your choices to pay will seem worth it. The distortion makes staying up late seem more worth it at the time than it really is taking the outside view.
So let’s say that when I would only pay $5, I regret it 60% of the time, and when I would pay $10, I regret it 40% of the time.
Then in Scenario 1 at X=5, I stay up and regret it 13% of the time and stay up and don’t regret it 12% of the time. At X=10, I stay up and regret it 4% of the time and don’t regret it 6% of the time.
In Scenario 2 at X=5, I stay up and regret it 11% of the time and stay up and don’t regret it 14% of the time. At X=10, I stay up and regret it 8% of the time and don’t regret it 12% of the time.
Now, how do we measure awesomeness? Let’s say it’s measured by the proportion of the times you don’t regret your choice minus the proportion of times you do (i.e., weighting them equally).
In Scenario 1 at X=5, awesomeness is -1%. At X=10, awesomeness is 2%.
In Scenario 2 at X=5, awesomeness is 3%. At X=10, awesomeness is 4%.
So the revenue-maximizing value of X maximizes awesomeness in Scenario 2 but doesn’t in Scenario 1.
Ah, nice! This sounds right to me and I don’t think I can seriously defend a claim that maximizing awesomeness and maximizing revenue are always and necessarily the same. I think the striking thing is that they correlate at all. And as a practical matter, defining awesomeness is actually quite hard. I like your use of regret as a measure; that’s probably the right ground truth measure. But if we don’t have access to or don’t trust surveys about regret (I personally might have a really hard time answering that question for my own past behavior!) and we still need to operationalize awesomeness… it might be hard to improve on “amount paid to Beeminder” as a proxy.
I really think this depends a lot on the goal. For instance, @narthur writes about his work commitments goal:
So in that case, the ideal is to never derail, and there’s no correlation between awesomeness and revenue for one derailment, but that changes if we look over the long term.
It’s worth looking at different types of derailments:
- For a goal where you never want to derail, but you derail due to akrasia.
- A goal where you set it up so that you can choose to pay $X to derail, you don’t choose to pay, but you derail anyway due to akrasia.
- A goal where you set it up so that you can choose to pay $X to derail, you choose to pay, and you decide in hindsight, taking the outside view, that it was worth it.
- A goal where you set it up so that you can choose to pay $X to derail, you choose to pay, and you decide in hindsight, taking the outside view, that it was NOT worth it - that is, you regret your choice.
With derailment type 3, there is a correlation between awesomeness of your choice and revenue. With type 4 there isn’t.
With types 1 and 2, the derailment was caused by akrasia rather than by your choice. So the relevant correlation with revenue is not single-incident awesomeness, but rather awesomeness over the entire time period preceding the derailment - like your example of the $1000 pledge paid for UVIs being worth it for the 1000 previous days.
In that kind of situation, let’s look at what happens as we increase the pledge amount X. With very low X, it won’t be motivating at all, so very little awesomeness. As we increase X, motivation increases, so the number of derailments decrease. Finally, at a certain point, making X larger will have no additional effect on motivation but will continue to increase revenue. So for example:
X % derailment awesomeness avg revenue 1 50 50 0.50 10 20 80 2 100 5 95 5 1000 1 99 10 10000 1 99 100
Here awesomeness is the % of the time you do what you want, which is to always keep your commitment.
So under this model, awesomeness and revenue are correlated up to the motivation point.
Thanks for this analysis! I think we also want to model the phenomenon of making the yellow brick road too easy when the pledge is too high. I think that’s the big reason why “just risk so much you’ll never dare derail” doesn’t work.
But there are definitely cases where it’s more binary: you hit the obvious minimum or you don’t. And those are cases where awesomeness maximization means a huge pledge that you’ll definitely never pay. So max awesomeness at min revenue, sadly for Beeminder. But I guess that’s why we have premium plans!
I’d actually like to collect more case studies that violate my thesis here. I mean, I’d love to believe they’re rare and that maximizing revenue is unironically the best thing we can do for our users. But loving to believe something is a pretty big red flag so y’all should push back on this!
I mean, is this really a concern? I think this adds a whole new layer of complexity because it’s unclear what “too easy” means. I was assuming a fixed rate - I guess I just don’t see the rate changing as a function of the pledge. I just think the analysis is way too complicated if you add that in.
The rate is what the user puts in as what they want to accomplish - it seems a bit paternalistic to challenge that. We should define success/awesomeness in terms of the user meeting their defined goal.
Besides, even if they start with a low rate, they’ll increase the rate over time.
What makes you say it doesn’t work? I think it works once people commit - but I can see how people would be reluctant to commit.
I’m hesitant to say “you’ll definitely never pay.” No one is perfect, which is why my table maxed out at 99% success. Under that assumption, as you increase the pledge, awesomeness plateaus while revenue continues to increase.
Yeah, I can tell you really want to believe there’s a strict correspondence here. I think if you step back a bit, it’s really the other way around - the more useful Beeminder is, the more people are likely to continue to use it, upgrade, and refer others. So really, long term, the best way to maximize revenue is to maximize awesomeness.
But if you look at the choices that either the user makes or that Beeminder makes in order to maximize awesomeness for the user in the short term, those are not necessarily going to maximize revenue in the short term.
For instance, if a user pays more pledges in the short term, they’re derailing more, and Beeminder is less useful to them, and so they’re less likely to continue using it. So that’s more revenue in the short term but less in the long term.
Whereas if a user is paying fewer pledges, yes, that’s less revenue to Beeminder in the short term, but it’s more awesomeness, which means they’re more likely to continue using it, which means more long-term revenue.
I’m partly disputing this part. If they leave then it means Beeminder was less useful to them. If they stay then it means the opposite.
Agreed. I’m focusing on the case that a user is paying a lot in pledge and doesn’t leave, which implies getting a lot of value out of Beeminder. If we drive them away due to derailing too much then that minimizes both awesomeness and long-term revenue and is quite compatible with my whole revenue=awesomeness thesis.
This is the key thing I’m disputing.You’re saying it’s a needless complication, paternalistic, etc, to question the slope of the yellow brick road that the user chooses. There are certainly cases where that’s true. I’m claiming that more commonly, though, when a user is never derailing it’s more likely to be because their goal is stupidly easy than because they are being super awesome. And more importantly, if a user is derailing a lot, then it must be that they’re deriving a ton of value (else they’d quit) which means doing a lot more than they’d be doing without Beeminder.
In any case, we definitely agree on the question of long-term revenue, which is the more important thing anyway.
This is now a major motion blog post!
this blog post is so… you, @dreev, with you and all your extremey extreminess
so how about these scenarios that seem to me like counterexamples to “maximizing user awesome is equivalent to maximizing revenue”
Say I have a ukulele kicking around my house. I have a job. I have friends. I have children. I have responsibilities. I also have a vague ambition to play that ukulele around the campfire of a summer night.
If I set up a Beeminder goal to practice that uke, and cap it at $5, I’m now going to start picking up the instrument and strumming a little bit here, a little bit there. It doesn’t have to be an ambitious goal to work on my behavior. I like playing it. It’s just something that I let get bullied out of my regular day-to-day. I might derail it every once in a while, but I don’t have any interest in increasing the pledge. I don’t have any interest in increasing the time I spend on that practice either. Making the goal “harder” is actually kind of counter to my actual real-life-goals. I mean, I guess it would get me to that campfire faster, but I’m not trying to make a career of this. I’m not trying to start a band. I’m not gonna be busking in the subway. The amount of “awesomeness” that beeminder can add to my life around playing this instrument is rather binary. Doing it at all is already enough. I don’t want to make it a stressful “i have to get into juliard” kind of thing.
So in what way does “awesomeness” correlate with revenue here?
Or what about how for me, derailing is quite often failing. One of my most common derail scenarios is because I’m over capacity and I’m getting overwhelmed and things are falling through the cracks and I start derailing more and more often and what’s really happening is that I’m toeing burnout. It’s more like the slow memory leak before a core dump. So what do I do: I go archive goals and decrease slopes. So for me as a user, paying Beeminder lots of money means I dial it back.
What about here? How is revenue correlating with awesomeness here?
These are important counterexamples! For the case of ukulele practice, it’s true, I had in mind an “i have to get into julliard” kind of situation. Like all the PhD theses that have been written thanks to Beeminder (true story, for anyone just tuning in!). In those cases I think my claim holds up – if you’re not derailing then crank up the intensity!
That’s not the answer in your ukulele example, but mostly because there’s that much less awesomeness to be harvested from that example. That goal only has so much value and it makes sense that Beeminder will only be paid so much. Which means you’re right – inducing more derailing would hurt, not help.
On reflection goals like that probably are more common. Just that they also matter less than the life-changing ones. So I’m clinging to my claim that revenue=awesomeness is still mostly true, for the right definition of “mostly”.
As for your next scenario, I’m tempted to say that’s just because you have your goals so well dialed in. But maybe I could use your scenario almost as a case in point. (Ok, this next part would work better if you weren’t a Beeminder founder.) You sometimes have expensive derailments and that’s during periods of non-awesomeness. Sounds like an anti-correlation! But you keep beeminding and keep pushing yourself and the only reason you’re willing to weather those expensive derailments during the non-awesome period is because of how awesome the Beeminder-augmented you is the rest of the time. So arguably the correlation is still there.
Nonetheless, it still may count as a counterexample in the sense that any inducement to more derailing would be counterproductive.
If I stop trying to be so extreme for a minute, I could scale back my claim like so:
For life-critical goals where the user has not yet honed their commitments and dialed everything in optimally, the more derailing we can encourage the better. It entails envelope-pushing and discovery that further the underlying goal.
Or maybe I’m making a God-of-the-gaps argument and as people show me more counterexamples I may have to retreat further. I’m convinced there’s something to this though. Also it might make sense to decouple this extreme revenue=awesomeness claim from “derailing is not failing”. The latter is very much true in multiple senses. Like what we talk about in http://blog.beeminder.com/seinfeld (“beeminding beats streak-minding”) and http://blog.beeminder.com/beenice (“it’s ok to treat yourself to a derailment if the cost ensures you do so sparingly”).
Maybe the key to seeing Beeminder’s incentives as wholly non-perverse is to remember that derailing does not equal failing. Derailing is a lot of things — a kick in the pants, paying for an immediate break because the goal has been really hard, valuable information about how realistic the goal is — none of which have anything to do with failing. The only ways to fail are to quit the goal, to set the yellow brick road to something stupidly easy, or to cheat.
That’s also the key of the article: This presupposes a certain definition of “failing”. And then goes on to say that derailing is no such thing. In my opinion what failing is, is a very subjective question that can’t really be dictated by Beeminder, because it’s an opinion (and emotions-) based matter and might change not only from individual to individual but also from goal to goal, day to day, etc.
I know Beeminder is a company and needs or at least wants to have a set of applicable principles (a (predictive) theory of beeminder) to gauge stuff like revenue and user engagement, but I think this is also a prime case of removing unknowables by presupposing certain definitions and then working under the assumption that the unknowables now have become knowable. I would even suggest that many users couldn’t even articulate why something feels like a failure, which makes following @dreev’s definition even harder.
To say it in an image: The article claims to be a map of a territory but it’s actually a blueprint for a terraforming project, since it doesn’t describe the landscape which is unexplored. Instead it draws up a class of use cases and tries to rework the discursive landscape in which it acts (ergo: convince other people that what they see is indeed what @dreev describes if we only put a fresh coat of paint on it. And remove this detail. And add this extra thing over here. And so on.).
But also that’s basically how you get to a predictive theory of anything: You control the discourse by standardization (of definitions) and so on… So I don’t even know if it’s a bad thing in that sense, but we also loose complexity (and in this case a lot of it) by clearing things up in that way…
This is confusing short-term and long-term. If they’re derailing a lot, that’s high revenue short term, which might or might not mean high awesomeness. They might be derailing a lot but not deriving much value, in which case, yes, they would leave after a while, but in the meantime you have (short-term) high revenue and low awesomeness.
There are different kinds of awesomeness we can talk about.
First is “gross awesomeness.” Using Bee’s ukulele example, gross awesomeness is how much you practice ukulele.
But that doesn’t take into account the downsides, of which there are two:
First, the opportunity cost of all the other things you might prefer doing. As Bee points out, doing it too much will actually decrease awesomeness because it adds stress and takes away time from other things in your life.
And second, the pledge money you have to pay. If you end up practicing a lot, but paying so much in pledges that it’s not worth it to you, beeminder may get a lot of revenue, but the net awesomeness to the user is little or negative.
The user wants to choose a rate and pledge that maximizes net awesomeness, given their level of akrasia.
@matti: Dang, there’s a lot of insight packed in to that. Thank you! I’m chewing particularly hard on this “not a map of the territory but a terraforming project” concept handle you just gave me.
I think I’ve added a bunch of confusion talking about long-term vs short-term revenue, the joke about running away to Mexico, etc, but in the blog post and all my arguments here when I talk about “Beeminder revenue” I always mean overall, long-term revenue.
The case of high short-term revenue and low awesomeness is simply a case of low revenue in the important sense.
Now we’re just quibbling about how to divvy the consumer surplus. That question is always secondary to the question of social/economic efficiency.
Well if we’re only talking about long-term:
In my view the optimal amount of derailing for the user is none or low, whereas the amount of derailing that maximizes revenue for beeminder is somewhat higher.
Too much derailing means your pledge is too low or your rate is too high. But more derailing means more revenue for beeminder as long as it’s sustainable.
- There’s an exception: If the pledge is very low, more frequent derailing might actually mean less revenue than less frequent derailing at a higher pledge, but this isn’t guaranteed - it depends on how the user’s motivation increases with money. If there’s a sharp cutoff at the motivation point (say $100), user awesomeness is maximized when the pledge is just above the motivation point ($101), whereas revenue for beeminder is maximized when the pledge is either insanely high ($10,000) or sustainably below the motivation point ($45). $75 might not be sustainable, and $25 might be too low revenue.)
In my view, as derailing increases, once you’re past the point when you’re choosing to derail once in a while (beenice), awesomeness for the user starts decreasing, yet (long-term) beeminder revenue continues to increase - up until we get to derailing levels so high that they are not sustainable because the user quits out of frustration.
No/little derailing - could mean either high awesomeness because you’re at your motivation point, or medium awesomeness because your rate is below optimal.
— Corresponds to medium income for beeminder long-term.
Medium derailing - could mean derailing is by choice, but then your rate is probably too high, so low awesomeness. could mean your pledge is below optimal, so low awesomeness. Users will probably not get too frustrated, though, so this is probably sustainable. Could mean it’s not the right goal for you.
— Corresponds to high income for beeminder long-term, if sustainable.
Lots of derailing - probably means beeminder is not working for you. Pledge may be below optimal, rate may be too high, or not the right goal for you. Likely to not be sustainable and lead to frustration.
— Corresponds to low income for beeminder long-term because it isn’t sustainable.
Each user has a level and type of akrasia corresponding to a function of two variables - frequency of keeping your pledge is a function of the pledge and rate.
- f = f(p,r)
Beeminder revenue is pledge times frequency of not keeping your pledge.
- BR = p(1-f) = p-pf
For each user and goal there is an ideal rate i. The rate the user actually performs will be the rate they set times the proportion of times they don’t derail, or rf. The user wants this to be as close to i as possible, that is, to minimize |rf - i|. The user also wants to minimize payment to beeminder, that is, minimize p-pf.
So we can model awesomeness to the user as a(p-pf) + b(|rf - i|). The maximum awesomeness is 0. a is a negative constant corresponding to how much it sucks to pay each dollar to beeminder, and b is a negative constant corresponding to how much it sucks to do the task more or less than the ideal rate.
@dreev’s theory that revenue is proportional to awesomeness can be expressed as c(p-pf) = a(p-pf) + b(|rf-i|).
This can be rewritten as:
(1) n(p-pf) = |rf-i|
where n is the new constant (c-a)/b.
(1) essentially is a claim about akrasia - that akrasia works in a way such that users’ akrasia functions generally satisfy, or come close to satisfying, (1).
To make the math a little easier we can instead assume the user wants to minimize (rf - i)^2. We then have:
(2) n(p-pf) = (rf-i)^2
We can rewrite this as np-npf = r^2 f^2 - 2rfi + i^2
We can write that as (r^2) f^2 + (np-2ri) f + (i^2 - np) = 0
Which by the quadratic formula means f(p,r) satisfies
(3) f(p,r) = (2ri-np) / (2 r^2) + or - sqrt( (np-2ri)^2 - 4 r^2 (i^2 - np) ) / (2 r^2)
What would be really nice here is a Mathematica 3D graph of f(p,r) with sliders to change n and i so we can see what this surface looks like!
Really enjoying reading this conversation.
Just jumped in to say that I just hope, @dreev, that you don’t underestimate the flexibility of the product you’ve created.
I use Beeminder to solve all sorts of problems; and the value I derive from Beeminder for each is often slightly (or extremely) different. I feel like you’re stuck on the hard-core “do-as-much-as-you-can” type-goals, when there are many, many other ways to derive value from Beeminder, and to maximize real-life awesomeness.
Here are the use cases I’ve had for it that I was able to remember off the top of my head (examples aren’t necessarily my own):
- I want something that will poke me on a daily basis, a reminder I can’t (safely) ignore (hygiene).
- I want to be motivated to do more on a goal than I normally would (exercise).
- I want to track data (health metrics, meta goals).
- I never want to have even the option of derailing (addictions, hard-core commitments).
- I want to have the option of derailing in unusual circumstances using a pledge bright line (work or study hours).
- I want to maintain a low commitment on something that I just don’t want to die (side projects).
- I want to make sure I don’t forget to do something that I only need to do every so often (check oil, check fire alarm batteries, etc).
- I want to have the freedom of doing an activity without risking a self-destructive binge; just need a guardrail (gaming, YouTube, social media).
- I want to actively eliminate an activity from my life (addictions, negative health behaviors).
- I just want something that will encourage me to put in the activation energy of starting, but will leave me free to do as much or as little as I want beyond that (side projects, exercise).
- I want to start my goal out very conservatively and not feel forced into ratcheting it up until I feel ready (scary goals, exercise, side projects, anything).
I think the thing to remember here is that your definition of awesomeness (the user pushes themselves to the point where it’s almost not sustainable), while perhaps sometimes correct, is a very isolated definition. Imposing it disregards your users’ real-life context.
Your users have specific problems, specific real-life goals, specific needs. Beeminder is a great tool because I as a user can configure it to meet many of those needs.
I’m afraid if you took your philosophy to is logical end, you might be tempted to remove a lot of that flexibility.
What a great post! I agree completely, and that’s a great list of use cases. I want to go through that and make goals for each of those.
This in particular is a great one. I have a few goals in this category, and they’re super awesome for me because they’re so easy to meet. All I have to do is get started, and so I virtually never derail.
One example is my goal to do stretches for physical therapy. Yesterday I had to hurry to a presentation in the morning and I was exhausted in the evening, so I probably stretched for about 10 seconds each, which was sufficient to keep me from derailing.
If I had to do more than that, I probably wouldn’t have done anything and derailed. More revenue for beeminder, but less awesomeness.
And this brings up an important point - it’s the user who chooses the pledge and the rate (and the goal requirement), and the user should choose those to maximize awesomeness. Beeminder should help the user choose those settings to maximize awesomeness, rather than to maximize revenue.
(Example: this suggestion to automatically lower pledges after not derailing for a while - forcing this or making it a default would in my view lower pledges below motivation point, thus increasing revenue and decreasing awesomeness.)
So thinking more about this, my algebra was kind of misleading, because the only values that matter are the user’s choices for pledge and rate. There will always be values of pledge and rate for which awesomeness is low. If the user chooses poorly, then beeminder might end up getting more revenue even with low awesomeness.
This exactly. My suggestion for defining awesomeness is how close are you to doing the task at the rate that’s best for your life, minus the amount you pay beeminder.
I see derailing more than once in a while as a sign that the user’s settings are not optimized for this awesomeness.
My point was more that maybe peak awesomeness includes, perhaps by definition, the meta-skill of having the rate for that goal set just right, so that it takes into account the unknowables and so has just enough buffer that those unusual, infrequent things don’t derail you. So I was suggesting that there might be some perfect pledge amount for each goal that would be just scary enough that I wouldn’t let the buffer get too small to be derailed by something unexpected, but also some perfect rate for that goal such that I’d be doing as much as I could on it given the better forward planning that includes the unknowns. Seems like that would be peak awesomeness, but would also decrease revenue.
Though, like @bee pointed out, this presumes that every goal wants to be maximized to the highest (or lowest, depending on the direction) rate that is possible, which I agree just isn’t the case.
Right. And all of this is the logic behind the pledge schedule: That there’s some point at which you won’t be derailing anymore because of the size of the pledge and so, in the end, derailments (and revenue) will drop and yet awesomeness will increase. The principles behind the pledge schedule and those behind “increased revenue means increased user awesomeness” might be in tension with one another.
I agree. Part of the beauty of Beeminder is in it’s flexibility in the ways it’s inflexible. I get to decide exactly what my goals are and enforce that, and for some goals that’s, “Let’s see how much I can push this”, and for others it’s, “I just want to never have the option to not make contact with this activity over a given interval”.
And everything @bee said
There’s so much worth talking about in this post that I kind of want to have a whole separate DM conversation about it!
Weight loss. Dialing a road above 2-3 lbs per week can have potentially serious side effects. So getting to a healthy rate and doing well at it will not maximize revenue, but will maximize awesomeness. Also, having just a weight loss goal (an output goal) and no pledges connected to actions mean that you might still not be getting motivation about what clearly needs to be done at a time of action (assuming you’re not in the group of users intending to just not eat until on the road). And that can perhaps be generalized to say that it might be an indication, not that you’re on the verge of awesomeness, but that you haven’t realized that the goal isn’t set up in a way that you’re actually going to get what you want out of it.