I’m proposing what I’m initially calling a “Plain Data” integration - that is, a Beeminder data source that can make requests to an arbitrary URL that returns just a number, and uses that number as a datapoint for a goal. It’s a bit like URLminder, and after discussion with @dreev on Discord it might make sense as just an extension on URLminder, but for now I’m proposing it as an entirely separate integration.
For my personal usage, it’d allow me to keep track of my flashcards with Mnemosyne. I have two goals, one for “scheduled” flashcards and one for “unmemorized” flashcards. My scheduled flashcards goal is currently powered by Make, but it was frustrating to set up, has been hard to maintain, and only pushes data twice a day. My unmemorized goal is currently just manual data, since I didn’t want to go to the effort of setting up another Make integration. Both of them would be trivial to set up with this type of integration.
Ah, thank you so much for writing this up! (I bet against it in your market about it in order to add to your incentive, you’re welcome!)
One thing that has occurred to me is how elegant a generalization of the existing URLminder this could be. Like, suppose that URLminder asks for not just a list of URLs but what metric to glean from those URLs.
Brainstorming:
raw wordcount (status quo)
first raw number that appears
sum of all numbers that appear
We have lots of use cases for 1 and @gbear605 has a great one for 2. Of course coolest of all would be to somehow specify an arbitrary function, but let’s not get ahead of ourselves!
A different “function” that might be helpful would be a simple JSON path, or maybe even something like JsonLogic to parse it. A lot of APIs might return something like {“my value”: 123}, and it would be nice to have some way to get at that. I guess first raw number would technically work for that, but a slightly more complicated API (eg. from a pedometer for “all time” data, {“steps”: 60000, “floors”: 703}) would need a way to get at a specific value.
That all gets more complicated, but if you’re already supporting multiple “functions” then it might not be so bad.
What if the “function” were a regex that transformed arbitrary text/html into a string that can be eval’d as arithmetic (not necessarily to be exposed that way in the UI)?
Then case 3 is just s/[^\d]+/\+/g. Cute!
But as @skorytnicki pointed out, we have some concerns from @adamwolf on this (even the easiest version) that means we’re not quite at green light for shipping something, sadly.
While I agree what Adam outlines would be the “gold star” version of this, I wonder if there are ways to reduce the risk to a tolerable level without doing all that work.
Some ideas, though you can probably come up with better ones:
You have to ask support to turn this on for your account
It’s only available in an expensive paid plan
You have to pinky promise to accept all derailments and not contact support for issues or call non legit
You can also delegate the work of adding a datapoint to a client. So beeminder would just ping the URL without waiting for response or scraping the datapoint.
It’s worth mentioning that it’s perfectly doable right now, but you have to create your own beeminder app. It’s simply a bit steep. How about we create a template of a Vercel application (or AWS Lambda, or…) to facilitate it? I imagine scenario, where you just have to do:
Part of my hope with this is that it could just be plug+play, without needing to run or host anything Beeminder specific, but something like a Vercel template that you can clone would be a decent middle ground at least.
Sliiightly offtopic (I haven’t used mnemosyne since maybe 2008 IIRC, so forgive me if this is obvious to everyone else): Is there an endpoint we can pull data from for users, or is that just an example for the spec?
Good question! There is an endpoint now. I added it a couple years ago, to use for Beeminder with a Make integration. I love that that’s a possibility with open source software, though admittedly it does leave a question of how applicable this integration type would be for other data sources that don’t have an endpoint and can’t just have one added.
The Make integration I set up kind of works but I’ve had a lot of frustration with getting Make to do it correctly at the right timezone, especially when I’ve changed timezones and had to update:
The Beeminder goal due time
The Mnemosyne start of day time
The timing of Make’s run to push data to Beeminder