The excluded middle

A week ago I posted a link to this article, written by David Frum, to my Facebook profile. There are parts of his analysis that I disagree with, of course, but on balance I found the article to be a thoughtful, constructive, and pragmatic take on heathcare reform from a conservative perspective. Most appealingly, Frum didn't engage in the petty histrionics of the tin-foil crowd: his article delightfully misses the entire {'Marxist','bloodless coup','facist','government takeover','armageddon'} set. I was impressed enough to check out his blog, where I found a collection of interesting, articulate political pieces written from a more-or-less conservative perspective. I also found a relatively intelligent commentariat who, in spite of ideological differences, manage impressively civil disagreement. I thought to myself that, in an age of Limbaugh, Beck, and Palin, Frum is exactly the sort of thing the political right ought to promote: reasonable, self-critical, even academic arguments for conservative ideas without anti-intellectualism and scorched-earth demagoguery.

Alas, it was too good to be true. On Tuesday, two days after the publication of the linked piece above, Frum was fired from his position at the conservative American Enterprise Institute.

I won't lie: I was really angry when I found out about it. Of course I can't say whether or not Frum was fired over his politics (if you like you can read his take as well as that of one of his AEI colleagues), but in any case it's disheartening when a political group rejects its moderate elements. People like to complain about the hulking inagility of the two-party system, but one of its chief benefits is that it discourages extremism. Radical ideas are first taken up by third parties, and if they become sufficiently mainstream they are picked up by one of the major parties. This forces the ideas through an incubation period, moderating them before they have any chance at becoming policy. This system works relatively well because the major parties have incentive to appeal to as large a base as possible. Empty political slogans or not, Reagan's big tent, Clinton's third way, and even Bush's compassionate conservatism sought common ground among an ideologically diverse electorate, thereby forcing moderation on the relevant party.

But this system doesn't work if a party caters to its extreme elements. If it continues to pander to the tea party bloc and push out moderates like Frum, the Republican party will give its stupidest and most reactionary elements control over its agenda, which is bad for everybody involved. Energizing a narrow, vocal portion of the base may garner short-term political capital, but it's a losing strategy in the long-term—one that poisons the political atmosphere in the meantime. White-hot partisan noise deepens divides while alienating moderate voters, and it takes more than angry paranoiacs to win elections.

My own politics are a hodgepodge of left- and right-leaning ambivalences, but my loyalties are beside the point: regardless of the party in power, we need a strong, moderating opposition. But when the opposition chooses downtown Glennbeckistan as its ideological epicenter, it relinquishes its claims to credibility and does real damage to democracy.

Here we come a-qwantzling

I've made it no secret on this blog that I love Dinosaur Comics. I find the strip extremely funny—both intellectually and viscerally—in a way that I can't properly explain to people who don't share my appreciation. If you aren't familiar, you should give it a fair hearing. It could change your life for the awesomer.

Ryan North, the writer of Dinosaur Comics, embedded a puzzle in one of his recent comics. Inspired by the cryptographic messages of early modern scientists like Newton and Hooke, he encoded the strip's punchline as an anagram: "12t10o8e7a6l6n6u5i5s5d5h5y3I3r3fbbwwkcmvg", meaning that there are 12 't's, 10 'o's, etc. He left it to his readers to decode the scripts and offered prizes for the first person to return the correct punchline. That was over a week ago.

So far no one has solved it. It's like Excalibur. Or the Riemann hypothesis.

Realizing that his "qwantzle" is challenging, Ryan has been slowly giving out clues. So far, this is all we know:

1. The solution is a single, reasonably grammatical sentence that fits the context of the strip. It begins with the word "I", contains a colon and a comma (in that order), and ends with a double exclamation mark!!
2. Letters in the solution are capitalized as in the code, and there are no proper nouns; thus, combined with the first clue, all instances of capital I must be the word "I".
3. All words in the solution have been used previously in Dinosaur Comics. (DC is searchable, and readers have put together a dictionary of all possible words. My untrimmed dictionary has 14,000 unique words.)
4. The longest word in the puzzle has 11 letters, and the next-longest word has 8; these words appear sequentially in the solution.
5. [RN recently posted a final clue: the largest word is 'fundamental'. It helps, I suppose, but I think most people had already guessed that, and in any case the search space is still obscenely large!]

Even with the clues, qwantzle is maddeningly hard. Naively, there are 97! letter combinations, and even if you incorporate all of the hints the number of possible word combinations is staggering—far too many for a computer to enumerate. So there's a small community of readers working on heuristic approaches to the problem, trying to combine human intuition with brute-force computational strength. But so far, most solutions (interestingly, readers have submitted many grammatical sentences that meet the criteria, but none of them has been correct) have come simply by guess-and-check.

I won't lie: I've spent more time than I care to admit on qwantzle. I've taught myself a new programming language and spent a few idle hours crash-coursing on computational linguistics. To show for it, I've developed two approaches that I thought were clever. One takes a valid solution, randomly deletes a few words, and forms a new anagram with the deleted words; this gives you a way to automatically explore variations on a solution that you think might be pretty close. The other performs a genetic algorithm on letter ordering alone. The letter orderings are ranked according to how well they correlate with DC dialogue, randomly mutated and crossed over, and made to compete in a pseudo-Darwinian process intended to improve the overall quality of the solutions.

But, despite (what I consider) reasonable creativity, these approaches don't work all that well. They WILL spit out technically valid solutions, but they aren't terribly grammatical. My next step should be to include natural language processing techniques—NLP is a new field with surprising success at computationally characterizing language as it is spoken and written—but I'm having a hard time being optimistic. In general, computers are far inferior to human brains at pattern recognition, and I struggle to believe that a computer could be made to recognize the right answer even if it found it.

I'm certainly on the lookout for new solution ideas, of course. But I think that the problem will remain unsolved until Ryan North finally gives out enough clues—at which time it will be solved by a human brain performing (computer-aided) guess-and-check.

[Note: I originally mistyped the anagram, so if any of you were working on the puzzle using my copy—and I hope you weren't—I'm very sorry and it's fixed now!]