Smart Apostrophes: They’re a Problem (in URLs)

Recently, The American Prospect published an article excoriating the “men’s rights” movement. It was a pretty good article, and well-received. Lots of people tweeted links to it… or, they tried to.

Curiously, those tweets all broke in the exact same way, pointing at a truncated version of the correct URL. That’s because the next character after the end of that truncation was a “smart apostrophe”, or a right single quotation mark.

And when it hit Twitter’s automatic URL-shortening service,, that service didn’t recognize ’ as a valid URL character. It decided that must be the end of the URL. Hence the truncation.

My reading of RFC 3986, §2.5 is that a really good implementation should have spotted the high-range Unicode character and encoded it as %E2%80%99, leading to the URL: And indeed, when I handed people that URL, it worked beautifully!

In short order, The American Prospect had put a redirect in place. Now, the “smart apostrophized” URL automatically pushes through to a new version that simply omits the apostrophe altogether — much like the WordPress “slug” for this post itself. (The sharp-eyed among you may have noticed that this post’s title is a self-demonstrating article.)

However, it was a little embarrassing for a while, when even using the “Tweet” button at the top of the article — a thing that looked awfully professional and well-tooled — would still result in the truncated URL and a 404 page.

So, What Can We Learn From This?

  1. Don’t use unusual characters in your URLs in the first place. Seriously, avoid them. Again, WordPress has made this super-easy for years; its slug-making routine strips pretty much everything that isn’t plain low-ASCII. (Of course, if your entire title is non-ASCII — say, you’re a Japanese site and your title is something like 狐は、何を言いますか。 — then the results may be idiosyncratic, at best. You’ll need some other method.)
  2. Beware of third-party tools. One of the things that stymied people’s ability to share the Prospect’s article was that the “Tweet” button had code that misread the URL’s terminating character. But The American Prospect didn’t write that code themselves; they were using AddThis‘ social-sharing buttons. And that’s actually a very sensible thing for them to have done: This is what third-party code providers are supposed to be for. But in this case, their code wasn’t quite ready for what the Prospect threw at it.
  3. Stay on top of what’s happening with your site. All things considered, this was a pretty small problem — it only lasted about a day, and particularly didn’t last for very long after the issue became clear (in my own corner of the Internet, at least, where I saw a fix only a few hours after I started seeing complaints and confusion about the issue). This is almost certainly because someone was paying attention, whether to social media, server hit logs, emails, or some other channel.

Those are, of course, aimed at site operators. For people like the coders of AddThis — or anyone else making library code — I’d just reiterate the usual advice to be sure your tests cover lots of different cases, and especially edge cases! And read the relevant specs to be sure you know what you’re doing; don’t just wing it.

There are probably other lessons to be learned from this. If you’ve thought of any, let me know in a comment.

How Do We Stop the Spying?

I should really write something about the ongoing — and increasingly horrifying — revelations of NSA spying. The latest round of news basically boils down to:

The NSA can now say, “I’m in ur crypto readin ur comms — all of them!”

Every time I try to write about this, my heart pounds faster and I feel overwhelmed by sorrow, and fear, and disgust, and I just want to go puke and then curl up in bed with the covers over my head and a nice, hot spiked with enough alcohol to anesthetize myself.

I don’t want to live in an Orwellian dystopia. But I do. I live in a country where the government can monitor everything, and there are surveillance cameras everywhere. Instead of 1984‘s “telescreens”, it’s cellphones and CCTV cameras.

I feel like those of us who have been advocating in favor of privacy, and in favor of citizen oversight of government intelligence agencies, for all this time are now very much in the position of the bearded gent in Randall Monroe’s three-year-old comic, “Infrastructures”:

But just saying “Hey, we warned you” is not enough. The fight is not over until we all give up — and we’re not about to do that.

How Do We Fight Back?

If you’re technically inclined, Bruce Schneier has some tips on how you can try to secure communications, at least to some degree. He also has a call to arms for engineers — if you only follow one link from this post, make it that one.

There’s also a very nice — if depressing — backgrounder on what the heck all this means by Johns Hopkins University cryptographer Matthew Green. At my level of understanding and education (which is to say, knowledgeable web developer, but with only minor knowledge of crypto systems), I found it very good for filling in some of the gaps and giving me a “what does this mean for me?” understanding.

Are There Political Remedies?

I don’t know what we can advise non-geeks to do at this point though. Obviously, the NSA is completely out of control. I have doubts that any political process can effectively rein it in — after all, we denied them authorization for the Clipper Chip back in the ’90s, and they just went ahead and got themselves the same result anyway.

So, as long as they’re around, our freedom isn’t safe. The way is clear: Abolish the NSA. If we can’t manage that, at least cut their funding to zero.

This doesn’t even have to wait for the next election, just the next budget cycle. Tell your legislators now: De-fund the NSA. And while we’re at it, prosecute those who have violated the Constitution and the will of the American people. Hold them accountable, and put them in jail. We can start with Director of National Intelligence James Clapper, who has demonstrably committed perjury in front of the Senate Intelligence Committee. But that’s a start, not an end.

Privacy, freedom, and surveillance need to be major issues in elections. If a candidate won’t take a firm stand for them, then don’t vote for them. Make these things issues in the primaries, where there’s a chance to get new blood into office.

“But what about terrorists?” you ask? I’m not nearly as worried about a terrorist attack as I am about what the NSA, and the rest of my government, are doing.

I’m more scared of my government than I am of the terrorists.

I don’t want to live in an Orwellian dystopia anymore.

Stop Designing For Men

If you’re designing or creating content on the Internet today, you need to remember one simple fact: Your audience has two genders, not just one. It isn’t just men, it’s women, too — and the women outnumber the men.

Take a look at this report from last year, on “10 Key Trends From the Banking Trenches“. On slide number 27, it displays the results of a Nielsen study on US digital consumers.

The big take-away is right in the green dot in the middle: Women equal or outnumber men in every category except one. They make up 54% of social networking visitors (so men are only 46%), and they’re also beating men in video consumption by 53% over 47%. They are tied, dead even, for smartphone ownership. The only realm in which men are a bigger slice of the pie is in tablet ownership, where they reverse the 53/47 split from video viewing.

If you think tablet owners are as big a demographic as “people who view video online”, you have a serious problem. (Namely, you’re completely unaware of current Internet trends.) How much traffic does YouTube get every day? Then add Vevo and Vimeo to that… Read More »

Why I Just Uninstalled Ad-Aware

I recently uninstalled Lavasoft’s Ad-Aware Antivirus. As part of the uninstall process, it suddenly took me to a page on Lavasoft’s web site asking me about what made me uninstall their product.

I consider that kind of rude and unexpected, but since I was uninstalling the product specifically because of my deep dissatisfaction with it, I decided I’d do the company a favor by filling out their feedback survey, from the link at the bottom of the page. On the second of 3 screens, it asked me to “Please explain the reason why you have uninstalled Ad-Aware”.

In a fit of brutal honesty, I wrote this:

It kept annoying me, even when it hadn’t detected anything. It should be unobtrusive, not constantly saying, “Hey, I’m installed! I’m guarding you against stuff! Oh, and hey, I just downloaded new threat definitions! Do I get a cookie now, huh, huh? Please pay attention to me!”

Obviously, it is pretty heartfelt. Whether you consider it justified or too-snarky, it’s still a genuine user reaction, and one that led to an uninstallation born of frustration and aggravation. (Ironically, I suspect Ad-Aware’s ridiculous level of own-horn-tooting was an attempt to keep users from abandoning the app on the basis that “it never does anything”. Still, if that was the idea, it sure failed, at least in my case.)

If you’re making software, consider when and how it should annoy the user — if ever at all. Mostly, you should aspire for your software to get out of the users’ way — preferably, to the point that your users no longer even notice the software itself.

Singularities Aren’t Just In the Future

In my first post about the Singularity, I rummaged through various possible definitions for “what the hell does ‘Singularity’ even mean, anyway?” On my list of five options, number 2 was:

“A time when when technological progress goes so fast that we people before it can’t predict it (or what comes after it).”

But this means that the Singularity has already happened. In fact, there have been dozens of singularities in humanity’s history — and they point both ways.

Singularities in the Past

I sometimes like to imagine what it might be like to snatch Benjamin Franklin’s mind from 1790, a few weeks before his death, and resurrect him here and now in a cloned, rejuvenated body. What would he think of modern life? How much would he understand of our technology? Of our daily life? Franklin was one hell of a smart guy, a renowned scientist and inventor (in addition to other careers). I think that, given some time, he could understand the advances we’ve made in science and technology at least as well as the average modern person. (Probably better than many, in fact.)

But how long would it take before he could really participate in our society? How would he react to modern clothes? Modern music (be it rock, hip-hop, or even “older” stuff like jazz)? Would he ever be able to really enjoy it? Or would it seem as strange to him as gamelan or koto music or the minimalism of Steve Reich does to most of us?

And then there are social roles. Would Franklin ever adjust to the idea of women wearing miniskirts and being legislators, cabinet secretaries, and CEOs? Even if he took to the idea enthusiastically, would he be able to casually interact with modern women and ethnic minorities without accidentally giving offense? Read More »

The Implications of “No Local Storage” Computing

At, Rob Pike talks about how computing should be everywhere, part of the infrastructure. He says storage “should be someone else’s problem, one I’m happy to pay to have them solve”.

But the problem is, when you abstract away a problem like that, it will come around and bite you later. The people using Megaupload found that out: They paid Megaupload to handle their storage for them. Then the US Department of Justice seized the entire domain, and innocent users who were storing their own data on Megaupload’s servers had to sue to try — so far, unsuccessfully — to get their own data back.

Somehow, I doubt that’s the kind of experience Mr. Pike is encouraging.

Services like iCloud and the Google suite of cloud products are pretty close to what Mr. Pike describes. But as Slate’s Tienlon Ho describes, the stuff you put on someone else’s servers can evaporate at any moment — like a cloud dissipating on a sunny day. Ms. Ho even claims that In the same notice informing me that it had disabled my account, Google told me for the first time that it reserves the right to terminate your account at any time, for any reason, with or without notice. Okay, perhaps they hadn’t used that exact phrasing before. But in her very next sentence, Ms. Ho links to Google’s Terms of Service, which do include lines like “[Google] may add or remove functionalities or features, and we may suspend or stop a Service altogether” and “You can stop using our Services at any time, although we’ll be sorry to see you go. Google may also stop providing Services to you, or add or create new limits to our Services at any time. (emphasis added)”

But who reads Terms of Service pages that closely? Honestly, even if I was surprised by Ms. Ho’s initial ignorance about the way Google (like any cloud storage company) “reserves the right to take away or vaporize our data for any reason”, it’s still a good illustration of the way an average user sees things. You may be storing your stuff on Google’s servers (or Apple’s, or Dropbox’s, or…), but it’s still your stuff. They should have to give it back to you, whenever you want it!

To get back to Mr. Pike’s dream: He doesn’t seem to be looking at the “what happens when someone takes it away” situation. Then again, that’s somewhat covered by his later analogy with the phone system: “Twenty years ago, you expected a phone to be provided everywhere you went, and that phone worked the same everywhere…. You didn’t carry a phone around with you; phones were part of the infrastructure.… [Similarly, the] world should provide me my computing environment and maintain it for me and make it available everywhere. (emphasis added)”

If it’s part of the infrastructure, then maybe — like the phone system — not only broadband access but even cloud storage should be considered a public utility, and regulated as such, with access guaranteed. But if that’s what Mr. Pike is assuming, then that assumption should be made very explicit. The alternative is a world where you don’t actually own any of your own data.

To Stop “Six Strikes”, Declare Broadband a Public Utility

Earlier this month, major ISPs started their “Six Strikes” program. By any rational measure, it’s a horrible idea. Yahoo! dubbed the plan“six strikes and you’re screwed”. Comcast’s implementation, using browser alert pop-ups, has been described as “a security disaster”.

Among other problems, it costs nothing for a copyright holder to file a complaint or accusation, but filing an appeal costs the user $35, making the system ripe for abuse.

Copyright and commercial lawyer Andrew Bridges calls the Six Strikes plan “Soft SOPA”, saying: “We have the government putting pressure on advertising networks and… payment processors, unofficially, to take the same measures that SOPA was going to require them to [do]. But now it’s a sort of ‘if you know what’s good for you, could you pretty please, wink-wink’ method.”

Obviously, this needs to be changed. And there may be a way to do it: Declare broadband Internet a public utility.

It’s an idea whose time has come. For one, the United States wouldn’t even be leading the way in this move. Internet access has already been declared a legal right, either by legislation or by court decree, in France, Spain, Finland, and Germany. The United Nations agrees, having issued a report in 2011 stating that they are “alarmed by proposals to disconnect users from Internet access if they violate intellectual property rights”. The report specifically calls out “graduated response” laws such as France’s “three strikes” law and the UK’s Digital Economy Act 2010. Read More »

Can jQuery Put Pressure On WebKit to Fix Bugs?

If you didn’t already know that Opera Software decided to toss its Presto rendering engine in favor of WebKit, just stop reading this post right now and go catch up on the past week of news in the world of web development and browsers. Don’t worry, I’ll wait.

Assuming you did already know that, you should check out jQuery Foundation President Dave Methvin’s reaction, “The Tragedy of the WebKit Commons“. He says “jQuery Core has more lines of fixes and patches for WebKit than any other browser. In general these are not recent regressions, but long-standing problems that have yet to be addressed.” He goes on to suppose that “Opera probably doesn’t have [much] incentive to fix the common bugs… — especially when jQuery continues to cover up these mistakes. Instead, Opera will want to focus on… features, all while people complain about jQuery’s “bloat”.”

It’s a reasonable worry, given some of the other things Methvin notes about how long some WebKit bugs have been around. Really, go read his entire post, too — it’s only 5 paragraphs long. And it means you don’t really need to read Stephen Shankland’s news article about it on CNET News — though his summation of it as a JavaScript expert saying: “WebKit, get your bug-ridden house in order” is amusing.

The real reason I bring up Shankland’s piece is because of a comment on it by one SteveW928, who points out that “In an odd way, JQuery is enabling the problem” — much as Methvin noticed that “jQuery continues to cover up [WebKit’s] mistakes”.

But the cool part — the real reason why I’m writing this — is the bit where SteveW928 goes on to say:

While, as a user I’d hate to see it happen, the best way to get this fixed would be [for] JQuery to pull the work-arounds and then, somehow, properly point the finger at the WebKit folks. (emphasis added)

I had to add that emphasis, and particularly on the word “somehow”, because I understand why that isn’t at all easy. (It’s the same reason Microsoft has so much special-case code in IE for Quirks mode rendering.) If jQuery simply declines to paper over a bug in WebKit, users won’t think “WebKit is screwed up; it doesn’t do Thing X right”. Instead, it’ll be: “jQuery’s broken; it doesn’t work right in WebKit browsers”.

But here’s a suggestion for what jQuery could do: Make two versions available — one with WebKit fixes and one without ’em. And let people see what the size difference is between them both.

Obviously, no one in their right mind will actually deploy a version of jQuery that doesn’t work with WebKit browsers. But that doesn’t have to be the point. Maybe by showing how much of its code is devoted to working around WebKit’s bugs, jQuery can effectively “name and shame” them — and bring focus to the need to improve the codebase.

In fact, they could do one for each major browser… let us all know just how wonky each rendering engine actually is. Comparing file sizes on the download page would be far easier than downloading each un-minified version and grepping for comments.

Commandments For Handling Passwords

If you’re taking passwords from users, here are some commandments you need to follow:

Don’t Impose a Maximum Length Limit

This is one of the most critical. One of the best things anyone can do to make their password — or pass phrase — more secure is to make it longer. Increasing the number of characters means an exponential increase in the time it will take to brute-force it (all other things being equal). If you impose, say, a 12-character maximum on someone who wanted to use 13 characters, you’ve just stopped your user from making their password at least 26 times harder to break.

You can’t possibly be doing this for security’s sake! The only other excuse I can imagine is to conserve space in your database. But since you’re hashing your stored passwords anyway (you damn well are; see below), they’ll all be the same size no matter what, so that doesn’t wash, either. This restriction makes no sense at all. Just drop it.

Apparently Microsoft is imposing a 16-character maximum limit on its online passwords — but I don’t recall anyone ever accusing them of being a paragon of security best practices. Their claim that “our research has shown uniqueness is more important than length” doesn’t change the fact that length is important, and limiting it reduces your users’ security (and therefore yours, too).

Don’t Tell Me I Can’t Use a Certain Character

Any character. The more characters I can use, the greater I can make the key-space an attacker would have to search to brute-force my password. In short, if you tell me I can only use, say, letters and numbers but no spaces or punctuation, then you’re forcing me to make my password less secure than I’d otherwise make it on my own! Read More »

The Problem With “Objectify A Man In Tech Day”

Update: While putting the finishing touches on this post, I found out that its creator is cancelling Objectify Day, for a host of very good reasons. I think much of what I wrote here can still be useful, so I’m posting this piece anyway. I’m glad to see that the purpose I had in mind has been achieved — and in fact, Ms. Alexander has mentioned a variety of other problems and reasons that I hadn’t even considered.

The offer I made in my penultimate paragraph still stands, though.

I truly do hope the best for Objectify a Man in Tech Day, coming up this Friday (February 1st). I can see that it has positive aims, and I hope it succeeds. Or more to the point, I wish it would succeed — but I really don’t think it will.

What Is It?

The idea was started by Leigh Alexander, and she’s trying to bring attention to the way women are judged on their appearance, even in realms where it should be completely irrelevant. Her technique? Try turning it around. Sort of a verbal Hawkeye Initiative:

On February 1, whenever you tweet an article, quote, comment or video from a man, add a comment about their appearance or attractiveness — “Great article on Final Fantasy XII-2 from the always-gorgeous Kirk Hamilton,” for example.

This is by no means a new problem, and it’s not even one that’s news. For example, I mentioned something about this in passing in a post of mine over two years ago, where I noted:

…the way women in IT are constantly, and ruthlessly, judged on their appearance: A profile on a female geek or coder in a tech blog will immediately start garnering comments on whether or not she’s “hawt”, while geek women who simply include a photo or two on their personal website will find them hotlinked from all over the web and subjected to insulting commentary.

(Of course, you can’t make such claims without supporting evidence, because — as I’ve mentioned before — some people who’ve never experienced the viciousness of life as a woman in tech will complain that you’re just making shit up. The words “subjected to insulting commentary” originally linked to Telsa Gwynne’s “Where did the pics go?” FAQ, but she’s since taken her entire site offline, so this is the best I can provide anymore.)

Still, that two-year-old post makes me a Johnny-come-lately to this issue. Garry B. Trudeau was already writing (or at least cartooning) about this issue over 40 years ago. At least in the tech reporting world, we haven’t moved the needle on condescendingly sexist reporting in over four decades. Read More »