This started as a fun discussion in the apiary channel of the Beeminder Community Discord. I’ve edited it into a sort of essay with interjections from others.
More naming nerdery! I dislike the term “tests” (as in unit tests, test-driven development, etc; code that verifies the correctness of other code). What’s my beef with this perfectly innocuous and perfectly descriptive word? Let me count the ways:
- It’s too common and overloaded. You can’t search GitHub issues for things related to one’s test suite, you can’t grep the code for the word “test”, etc.
- My usual spiel about the value of jargon. See the “onomastic interlude” of blog.beeminder.com/brl.
- It connotes nonce-iness – something you do temporarily to try something. “This is just a test” etc.
Of course this is moot if I don’t have a better alternative, which I do: quals! Quals already means “test” in some contexts, as an abbreviation for qualifier. Like you have to pass your quals before you’re a PhD candidate, etc. It also evokes quality assurance, which makes sense for software.
In conclusion, the software industry should join us in using the term “quals” instead of “tests”!
More on the 3rd reason: The code and comments are full of “test” in the “checking something” or “trying something” sense so it’s genuinely valuable to clearly distinguish between tests like that and the actual quals, the components of your quality assurance / anti-regression infrastructure.
(@larsivi: I have worked a lot with safety and security software and hardware, and, well, test is what they do, even NASA. SUT and FAT and SFAT and whatnot.)
(@rperce: I use “spec” a lot. Which has the benefit of already being idiomatic in several popular test frameworks like jest and rspec.)
I considered specs but I think it’s too distinct.
(@rperce: Yes, specs are slightly more specific than tests in general, but I prefer them, so I’m fine with that. One benefit of writing specs is that then you get to call them that and it’s nice and greppable.)
(@larsivi: You can say “verified”. “Qualified” is a very space-hardware thing to say.)
Ooh, yes! How to verb it is a key question. But I think that helps make my case. In the “tests” world, the sentence “I tested that” is problematically ambiguous. Do you mean you just tried it or do you mean you actually qualsified it?
(@larsivi: qualitated!)
“Qualsified” being my tentative verbing but we could probably do better. Ha, right on cue, @larsivi with “qualitated”!
(@rperce: I’m ok with “qualsified”. But it’s critical to decide how you want your jargon to work in all the different parts of speech because otherwise you’re like a conlang with no morphology constraints where llwqg ends up being a perfectly cromulent word.)
Yeah, “spec” also solves all my problems, assuming I’m ok with the ambiguity between specs vs quals that aren’t technically specs because they’re like unit quals for functions or other kinds of quals that aren’t about specifying behavior. That could all be fine though, to just expand the definition of “spec” a bit.
But also I’m pretty into this definition of “spec”: blog.beeminder.com/specs. So that’s an unfortunate ambiguity for me.
(@rperce: Writing unit specs is perfectly cromulent already. And where’s the ambiguity? If you have a good functional spec for how your application should behave, a very reasonable-to-me place to write that down is in a file with enough context around it that a computer can validate that the application still meets the spec? I contend that those two degrees of technicality in spec-writing should live together and be thought of as different flavors of the same thing.)
(@larsivi: I worked with Fitnesse at one point, the selling point being that someone that knew the business rules could write tests for that part. Specified tests in a wiki format, assuming you had written the functions that could handle the data.)
As for the “spec” ambiguity:
- Definition 1: The document that states the goals or the problem being solved and how all the code will work from a user’s perspective.
- Definition 2: Code that verifies that functions given known input produce expected output, or that some sequence of UI interactions yields the expected stuff ending up in the database, etc etc.
But yeah, not necessarily crazy to overload “spec” for both of those.
(@rperce: I still think those are in many ways the same thing – in the first one the “user” is a human and in the second it’s another function or whatever, but the same design process still applies to both.)
Another stab at it: “A document in English specifying how future code will function” vs “Code to verify the behavior of existing code”.
(@rperce: Ah! Write your verify-code first, then it’s “a document in English specifying how future code will function” and “a document in Java specifying how future code will function”.)
(@larsivi: The first is a specification; the second a spec.)
(@rperce: Or rather than “those are the same”, those can be thought of as different things, but considering them to actually be the same leads to, in my experience, useful design and usage insights that improve both.)
You’re making a good point, especially in the TDD world where you actually do write the quals first.
(I’ve actually gotten used to calling them “quals” already; we’ve been experimenting with the new term for a while now.)
This is all hugely helpful, @rperce and @larsivi. So far I stand by my reasons to hate the term “tests” but “specs” may well be the less upstream-swimming replacement.
(@poisson: I am still shocked that “test” isn’t greppable. Is that really true?)
Definitely. Things like (to make something up) “this function tests whether the goalname the user typed is too long” which is wholly unrelated to “quals”.
(@rperce: I think if you can cement “quals” as a distinct word rather than an abbreviation in Beeminder’s culture it can work! It’s just not what I would choose greenfield. Curious, though: how often has the ungreppability of “test” in general bit you? Because while it’s definitely true, it’s not true in individual use cases: find . -name '*.test.*' works great, for example.)
Yeah, I don’t know if it’s caused enough actual grief to justify the work of changing terminology. At some point it annoyed me and I was nerd-sniped by the onomastic challenge, is probably the most honest answer?
More followed, including discussion of the term “story”, but this is probably a good place to stop.
, and