Clean and Beta Versions/Options, Please

Bugabee
1

I sent this privately on Nov 15th, but it seems relevant to this discussion so I’m posting it here with a few changes (made in square brackets) for anonymity:

Hi Guys,

I was wondering what you might think of having two interfaces. One at Beeminder.com and one at something like beta.Beeminder.com. Right now there’s no way to opt out of the beta group and there are occasional times when, speaking only for myself, anyway, I just feel like using a version that I can be sure is stable and just works. One that has fewer bells, fewer whistles, sure, but is guaranteed near-perfect. And then I could pop over to beta.Beeminder.com if I wanted to use a feature that was a little newer and I had the time to check and double check what it had done to my graph, do some bug reporting, play around a little, etc…

It’s really fun that you guys want to keep Beeminder fresh and that you’re ALWAYS working to add user-requested ideas, but there’s no way to be a user who is just a normal everyday person who comes in, uses a solid version of the site for its stated goal, and logs off. No one is able to opt out of the beta group. Not even first-day users who can’t tell the difference between I-don’t-understand-what-I-did and don’t-worry-that’s-just-a-soon-to-be-crushed-bug.

And, for me, introducing the site to others is, right now, out of the question. My [workplace is having a] competition and I’m doing the logistics of it. I thought of proposing we use Beeminder, but I didn’t want to introduce anyone to Beeminder at a time when the core pieces weren’t working. I want them to stay and play!! ([This was an example of a bug that got fixed in the last few days; I’m not going to replace it with a new example.]) Each instance alone is no big deal, but the fact that there’s always at least one core thing and a few side bugs is hard, especially on newbies.

Having a path to come in for a solid, known-good interface and a path for a beta, help-us-try-this-new-stuff-out interface would give users a choice. I think a lot of your long-term users would bookmark the beta, but newbies could be introduced to an always-working version, and on busy weeks, long-term users could use a no-need-to-double-check version.

The site’s been around a while, and gotten a lot of attention. To keep newbies, they have to trust you with their credit card (I know I keep banging that drum) and they have to trust you with their data. I think not being opted into a beta group would help with that and would launch the site into it’s next stage.

As always, these are just some thoughts. Feel free to ignore them!
[Chips]

PS - My tone isn’t intended as criticism, either. I LOVES ME SOME BEEMINDER ([…]). I want to share it with everyone and hound them until they try it. This is in service of that… and of wanting Beeminder to be around forever. Not at all because I don’t think it’s good or good enough.

2

Repeating my emailed reply from Nov 15:

Boom, just fixed the date-walking-forward bug. What’s the next most frustrating one?

(thank you so much for helping us like this; no need to emphasize that you still love us. just saying “current bugginess leaves me not recommending beeminder to people” is incredibly important for us to know!)

PS: still thinking about the clean-vs-beta proposal. @nick has a compelling counterargument in another thread…

1 Like
3

It is totally fair criticism that you (and others) have leveled at us lately for being sloppy (see also: arbitrary deadline buglist). We really appreciate the tougher loves.

Maybe it is worth pointing out, though, (so that newbees reading this don’t get the wrong idea) that your personal usage of Beeminder falls outside the norm and is like in the 99th percentile of bug-encountering usage. Using deprecated behavior like auto-quit, and then manual restarts which most users don’t ever end up doing. Even features like autoratchet, because to be honest if we did have a distinction between beta and core beeminder, all the _ratchets would still be in beta (simply because they have been so difficult to get right).

Keep the bug reports coming!

2 Likes
4

TLDR: Scroll down to the last paragraph.

Yeah I totally agree. That was originally an email sent directly to you guys, and it wasn’t phrased with a broader audience in mind, so that’s a good point. That said, while there are some bugs that are definitely caused by my slightly excessive use of the site for… absolutely everything, there are others that are affecting much more basic features. Features that it’s got to be the case that more people are using. And I may be odd in that I’m encountering very many of the bugs, since I’m doing so many weird things, but it seems likely that many users will encounter at least some of the bugs. The newer and less committed the user, the lower the bug-tolerance threshold. And not just out of frustration, but out of fear. Beeminder is different. It has to have different standards because doing things on the site, vs just subscribing to the site, can cost money. That changes the level of new-user nervousness. They have to be able to trust they they can figure it out, and they have to be able to trust that it will work right.

I still really think deprecating auto-quit is newbie-deterring. And if a new user does check autoquit and then encounters that bug, that will happen right at a decision point. Having just derailed their first goal and paid their first bit in, they will come to a point of deciding whether to continue or not… and they’ll be met with a bug. Also, how will they know it’s been deprecated? The knowledge needed for using the site should never depend on being in the forums and reading the daily or weekly emails, IMO. (And I do both and, though I knew you guys preferred we didn’t use it, I didn’t know it was deprecated until I read that just now.)

Right now there’s no visual distinction between “this has been here forever and works fine” and “use with some caution”. They all show up in exactly the same way and it’s up to users to remember what’s deprecated and what’s not, what’s working and what’s a new feature that might still be buggy, which bits of this feature work and which don’t, etc. I don’t think you can rely this much on users having as much advanced knowledge about the way the site works without making it new-and-moderate-user-unfriendly.

Maybe you could do something like hide any beta/buggy/deprecated features in a beta tab, instead of on a beta version. Or, perhaps a better idea: you could have it such that we could select whether to show/hide features, and you could default-hide any that aren’t newbie-friendly or that still need a little TLC. That doesn’t require maintaining two systems, but it makes it unlikely that newbies will bump into bugs in new features or will use features you think might be confusing until they’re at the point of exploring further. You could also have warnings on those features that have bugs flagged. Then make it an “everything stops until this is taken care of” kind of priority to deal with any bugs that come up in the “basic” features, the features that are always shown, the defaults. Make it such that everyone focuses only on those bugs if they crop up. That way, newbies are always safe, confusing extras are hidden from view (there isn’t the “holy crap… there are 4 pages of settings on each goal” overwhelm when people start out), and there’s a place to warn people about features that you wanted to give us, but that are still a little wobbly. You could even send a warning message to all of those who have unhid a given feature if a bug is discovered with that feature, so that they can choose to hide it or to use it while keeping that in mind. That way almost no one ever gets a surprise bug biting them in the rear. Meanwhile, people like me who will, let’s face it, unhide everything and break anything breakable, can continue using any advanced features you create and flagging bugs to you. All the spirit of beta without the technical problems Nick mentioned in the other thread. (Well, the show/hide thing would be a technical issue, but it’s probably less likely to cause graph-related bugs than the beta thing and I think it solves a number of problems in one shot.)

5

Also, this seems at least somewhat relevant as I get ready to get to work on things I should have started when I first got up, instead of posting here: http://xkcd.com/1445/

2 Likes
6

Have y’all considered beeminding the backlog of known bugs? Commit to knocking out a bug every two weeks or spending 5+ hours a week on debugging efforts (cause some bugs take awhile to fully figure out) or whatever. Makes the site better for newbees, and makes old users happy that the issues they found (through borderline-obsessive site usage) are finally being taken care of.

I know there’s UVIs and infrahancements already. Beeminder is really into new features and changes, which is great!! But it can have the annoying side effect of creating new problems & backburnering old ones, which is less great. The whole point of Beeminder is to do the things you say “we’ll get that done someday!” about, so why not track your bugs the same way?

(Semi-related rave: I really think inbox fewer is my favorite, most useful goal type. I love it a lot. It should be more widely used… maybe renamed so people know what it does!)

2 Likes
7

One thing that may help here is if we did a better job of conveying that anytime you go off track and we’re about to take your money, we email you. Just reply to that email if the derailment wasn’t due to akrasia (or however we’d convey that to newbees) and we’ll make sure you aren’t charged. If users trusted that [1] that might go a long way towards solving this problem.

Deprecated was the wrong word. It’s just tucked away in advanced settings to make clear that you shouldn’t set goals to auto-quit unless you understand what that means. Automatic rerailing is, we’re convinced, exactly what newbees need so they don’t derail once and then procrastinate indefinitely on getting back on the wagon. (And of course if you do want to quit, just hit the red Archive button. The real point is that quitting should be super easy to explicitly choose but shouldn’t be the path of least resistance. That’s our hard-won insight about the psychology at play here.)

Smart. Of course “terrifyingly advanced settings” signals this pretty well. We just need to reorganize what things are in that section.

Excellent advice, as always!

[1] And they can! We now have @greatandrandom, @quintrino, @mathir25, @mel, @jill, @pjh, @chelsea, @bee, @alice, and me all chipping in monitoring support@beeminder.com from all around the globe so we can jump on spurious derailments and answer questions.

1 Like
8

As you say, we do have UVIs and infrahancements, and I think those actually cover this pretty well. (See my other comment about the fraction of UVIs that are bugfixes.) My feeling now is that the only known bugs that we’re not jumping all over are things that will be fixed by bigger changes – like how arbitrary deadlines fixed the brokenness with submitting data between midnight and 3am or how the generalized road dial will fix brokenness with scheduling breaks and ratcheting.

I think we’re really good about not just shrugging off bugs. But keep hollering loudly any time you’re bitten by one; that’s a huge, huge help to us. I know some of you are super nice and don’t want to be whiny. Please suppress that instinct! (:

9

oh for sure! I can’t even think of one I’ve actually encountered myself - just the trickier cases that tend to pop up in support (multiple Duolingo languages, difficulties with ending/restarting goals, etc).

1 Like