Downtime?!

Hi all! The British workerbees are aware of a problem that seems to have taken Beeminder down, and have just got in touch with the powers that bee (meaning bee and dreev).

At the moment it’s unclear what the problem is, and it seems somewhat intermittent since we’ve briefly seen it come back up.

If you derail due to this (due to not being able to enter data, not knowing what you need to do to fulfill the goal, etc), then as ever, that’s our bad and you can contact us by replying to the legitimacy check email to let us know. For now, though, please settle down on a nice tasty-looking flower (unless you still have steps to get or walls to climb or emails to answer, in which case you can do that!), and let us do the buzzing.

6 Likes

Looks like pages are loading at the moment, whew! Investigations continue.

5 Likes

As an addendum, the support inbox is very, very full as a result (partly with alerts for stuff that failed during the downtime). Right now it’s just me working on that, so the queue is going down slowly but surely. I’ll call in backup if necessary, so just hold tight! We’ll get to your email as soon as we can. :slight_smile:

5 Likes

Hey all! Things seem stable right now. As far as I understand it, we don’t 100% understand what happened… some things have been tweaked, notes made, etc, and probably the investigation will continue. But the important part is that things do seem to be back to normal, or getting there!

The support inbox is also nearly caught up, onto stuff from just the last 2 hours (which is normal; our average first reply time is 6 hours).

So hopefully all is well now and you can all go back to buzzing, if you haven’t already. Our apologies for the disruption!

5 Likes

Hey everyone!

Unfortunately, it looks like we’re experiencing downtime again. For now, we’re not sure why, but as ever, hold tight and we’ll investigate. If you derail as a result of this, reply to the legitimacy check email in order to let the support team know, and of course we’ll cancel it.

It’s possible this isn’t something on our side (especially as nothing was being done on our end on a Sunday!), so without any more information than this, we can’t give an estimate of when we’ll be back up. But Adam’s on it, and we’ll do what we can to get the bees buzzing again ASAP.

4 Likes

We’re back! Looks like a bit over a half hour of downtime :grimacing:

4 Likes

Did you work out the cause?

1 Like

As I understand it (bearing in mind I’m the rubber ducky rather than the person who knows what’s going on), we were having trouble connecting to our database server, possibly connected to disk space. Monitoring continues, to see if we can confirm exactly what happened/whether it could happen again. I don’t think we understand why it would suddenly be a problem when it wasn’t before (but I could just be missing something that’s obvious to everyone else).

At the time, a reboot and some cleanup got things back to normal, but sounds like we’re looking at different hosting options for our database and whether there’s anything reliable/cost-effective out there that suits our needs better than the status quo.

Take all this with a pinch of salt, as I wasn’t officially working on the day, so I didn’t join the call and have only seen the summary. :slight_smile:

4 Likes