Word count tracking Python script

Tech
1

Inspired by Beeminding Markdown Folders (Obsidian) - #2 by april I wrote a Python script to calculate word counts from set of Markdown files matching a file glob and post the difference in values to a Beeminder goal: https://github.com/tguernier/beeminder-wc/tree/main

This was written for Linux but it should work on Windows as long as you have uv installed and you set the config to use Windows paths.

Some limitations/bugs of the script as it is currently written:

  • Word counts are inconsistent compared to Obsidian/other editors as this script just uses len(text.split()) to count words and the other editors probably do something fancier. Don’t think this makes too much of a difference though.
  • This only posts to Beeminder if the word counts for the entire glob has increased so it won’t capture editing work that doesn’t increase the word count of the file.
  • Maybe it could have something to post custom comments to datapoints depending on what files are edited?

I’ll probably fiddle with this script as I figure out my writing workflow but I’ll try to keep it as general as possible.

4 Likes
2

And now that Buzz supports piping in values you can do a simple word count over a set of directories and enter it into your Beeminder goal all in one line of bash:

find /path/to/your/files -name "*.md" -print0 | xargs -0 wc -w | tail -n 1 | awk '{print $1}' | buzz add wordcount
5 Likes
3

This looks great!

I had a complicated solution for a writing goal before which tried to use git commits to also track deletions, to get all changes. I feel like it didn’t work super well though, and I haven’t tried to improve it at all since since I haven’t made another writing goao.

1 Like
4

Since my last post I’ve made a few tweaks, the biggest one being using the Obsidian Shell Commands plugin instead of cron to execute the script automatically from inside Obsidian.

I also wrote a blog post about my whole setup: My Beeminder + Obsidian + Shell scripts setup | Furret Blog

2 Likes
5

Cool! Does that mean you’re posting to the Beeminder API every 60s, to update the values (potentially 1440 times per goal per day)? Or do you cache the last value and only send updates, or something?

(Of course the API can handle 1440 requests per day, but it seems excessive if you can do something more sensible from the client end)

3 Likes
6

It only posts values when the word count has increased, but it does poll the goal value from the Beeminder API every time the script is executed. I don’t have Obsidian running 24/7 so it’s not quite 1440 times per day, but it’s still once every minute when it’s open.

The shell commands plugin has an option to run the script on file changes and a debouncing feature to limit the number of times the script is executed, which could possibly cut down on the amount of times the script is called.

1 Like
7

I do something similar for several goals, but keep the most recent posted value in a local cache file so that I don’t have to call Beeminder unless that file is missing or is from the day before.

MEMORY="$HOME/src/bmndr/.lastcount"
TODAY="$HOME/src/bmndr/.today.date"

count=LOCAL_POLLING_OR_CALCULATION #TODO

# make a midnight comparison file
/usr/bin/touch -t $(/bin/date +%m%d)0000 "$TODAY"

if [ "$MEMORY" -nt "$TODAY" ] ; then
  # exists and is from today - check value
  lastcount=$(cat "$MEMORY")
  if [ $count -eq $lastcount ] ; then
    # no update required
    exit 0
  fi
fi

echo $count > "$MEMORY"
2 Likes
8

I figured out how to get the word count to update on file changes without hammering the site too much. In the Obsidian Shell Commands plugin settings:

image
image1494×136 16.5 KB

image
This only posts a value to Beeminder when words have been added to the file at a maximum rate of once every minute.

3 Likes
9

I also figured out how to run the script in GitHub Actions so I can run the script every time the Obsidian vault is pushed to GitHub (using the obsidian-git plugin).

I’ve got it set to push once every 10 minutes and it uses only a tiny fraction of the free GitHub Actions usage quota (~60 minutes out of 2000 free minutes running it). Instructions are in the repo readme: beeminder-wc/README.md at main · tguernier/beeminder-wc · GitHub

2 Likes