I found this working on the dashboard. The json library I use choked on the title and reports the above as
{
"slug": "water",
"title": null
}
and ends the json document there. To me that looks like a bug on the library’s side. The RFC does explicitly say characters MAY be escaped.
Still, I’m reporting this here because I think the escaping in the title is unintended given that a) it needs no escaping and b) other attributes are not escaped either.
I entered the title via the website in top part of the goal page:
Taking a look at what’s stored in the database, we’re storing the unicode un-escaped, and the api is not explicitly escaping things. I just ran a test with one of my own things, and other unicode characters are fine, but when I copy in the description from your water goal, it’s giving the escaped unicode with curl, and showing up unescaped in other libraries… so something with the json parsing / library on the other end perhaps?
Here’s what I see via curl "ϡ and ⎈ ⏰ ◒ \u0026 and hurray!"
and via my ruby library of choice "ϡ and ⎈ ⏰ ◒ & and hurray!"
That’s what curl receives and that’s what the library I am using receives
Your ruby library of choice then goes ahead and unescapes the \u0026 again before giving it to you
The lib I’m using recently added a flag to control this behaviour. It is disabled by default because it puts additional strain on the already tiny processor in an Arduino, I assume. Using this flag the \u0026 does get translated back to & just as it should and the library does not choke on it.
I wouldn’t know why the API sends out the ampersand escaped but I have a suspicion which I hope is wrong: It might be that they think they do you a “favour” by escaping the oh-so-dangerous ampersand so it doesn’t to funny things in your HTML by accident. I hope I’m wrong
Ah, I suppose that’s probably it. Anyhow, you’re correct; I was looking in slightly the wrong place previously, and yes, Beeminder’s JSON library is “helpfully” escaping just-that-one unicode character.
I guess that means “It is the opinion of this Authoritative Source [i.e. a widely used JSON gem that’s probably the default thing that most Rails use], that escaping ampersands is the Correct Behavior”, in which case, I’m loath to change that default behavior without good reason.
Totally agree. Either way, there is no “wrong” behaviour here as far as I see. It’s technically correct to escape all characters inside a JSON string as far as the RFC is concerned. If that’s desired or not is of course a different and subjective question One that ultimately depends on the… wait for it… character.
