Shtetl-Optimized

This is a question recently asked by Lance Fortnow. There are a few boring answers: I thought I wouldn’t have time, what with my packed schedule of websurfing, procrastinating, and sleeping. I thought the human race had already overpopulated God’s green blogosphere. I thought the bandwagon had already passed in 2003, and there was no use chasing it now. I thought it would be presumptuous (as indeed it is).

But the real answer is that to run a successful blog, I knew I’d have to write about what actually mattered to me — and that included more than just the latest arXiv preprints or bizarre complexity classes. I’d have to state strong opinions, make my worst fears everyone else’s business, probably offend some people, and probably embarrass myself. So before I did that, I wanted to make sure I could at least do it in the best, most eloquent words — words that couldn’t possibly be misunderstood.

So what happened? Did I find those words? As you can see for yourself, I didn’t. What happened is that, after finishing grad school and reaching an advanced age, I started to face my mortality. Before then, I could always justify inaction by telling myself I was still preparing for the rest of my life. But once you’re in the rest of your life, if you’re not actually living it, then what are you doing? It occurred to me that, if you wait for the “perfect opportunity” to start a weblog — or switch to a new research area, or ask someone out, or whatever it is you want to do — then you’re essentially just committing delayed suicide. I’m sorry if that sounds trite and obvious.

Efficiency matters. Time constraints change everything. How could I have forgotten?

Follow

This New Yorker piece by Malcolm Gladwell shows once again why, despite all the hype surrounding him, Gladwell really is one of the most perceptive social observers of our time. Gladwell is reviewing The Chosen by Jerome Karabel, which relates the history of the undergraduate admissions process at Harvard, Princeton, and Yale.

Have you ever wondered why that process places so much emphasis on sports, extracurriculars, personality, “leadership,” “character,” and suchlike, as opposed to the more obvious intellectual criteria? The answer, it turns out, is that in the early 1920’s, Harvard and Co. had to find some way to limit the number of Jewish admits:

By 1922, [Jews] made up more than a fifth of Harvard’s freshman class. The administration and alumni were up in arms. Jews were thought to be sickly and grasping, grade-grubbing and insular. They displaced the sons of wealthy Wasp alumni, which did not bode well for fund-raising. A. Lawrence Lowell, Harvard’s president in the nineteen-twenties, stated flatly that too many Jews would destroy the school … Finally, Lowell — and his counterparts at Yale and Princeton — realized that if a definition of merit based on academic prowess was leading to the wrong kind of student, the solution was to change the definition of merit. Karabel argues that it was at this moment that the history and nature of the Ivy League took a significant turn.

Gladwell writes that from that point forward,

The admissions office at Harvard became much more interested in the details of an applicant’s personal life. Lowell told his admissions officers to elicit information about the “character” of candidates from “persons who know the applicants well,” and so the letter of reference became mandatory. Harvard started asking applicants to provide a photograph. Candidates had to write personal essays, demonstrating their aptitude for leadership, and list their extracurricular activities … The personal interview became a key component of admissions in order, Karabel writes, “to ensure that ‘undesirables’ were identified and to assess important but subtle indicators of background and breeding such as speech, dress, deportment and physical appearance.”

The byzantine admissions process that Harvard and the other name-brand schools set up pretty much remains in place to this day. (I still remember the “Potpourri” section of the Princeton application, which asked applicants to list their favorite movies, music, etc. It reminded me of the bridgekeeper from Monty Python and the Holy Grail: “What is your name? What is your quest? What is your favorite color?”)

Granted, the Ivy League admissions process no longer serves its original purpose, possibly because most American Jews have become so assimilated themselves. But today, the enormous preferences given to legacies, athletes, and other students of “character” work very similarly to limit the number of Asians. To which I can only say: huzzah! For as Gladwell explains: “If Harvard had too many Asians, it wouldn’t be Harvard, just as Harvard wouldn’t be Harvard with too many Jews or pansies or parlor pinks or shy types or short people with big ears.”

Follow

To get this blog rolling, I’d like to put forward a modest idea that I’ve been chewing on for a while. Ready? Here it is:

Scientific papers are a waste of time. Therefore, we should stop writing them, and find a better way to communicate our research.

Among the likely readers of Shtetl-Optimized, I can’t imagine that this idea would cause the slightest controversy. But just in case I’m mistaken, let me stress that the idea would have seemed crackbrained to me, too, back when I was young and green.

“Are you kidding?” I would have screamed at my decrepit 24-year-old future self. ”Research papers have been humankind’s great instrument of progress for 300 years! They’re the bulwark that separates Crick from creationists, Chandrasekhar from Chopra, and Wigderson from wackballs like this! Without peer-reviewed papers, how would we verify each others’ claims? How would we establish priority? What would we fill our c.v.’s with?”

So I came to my current view slowly and reluctantly, as a result of spending the past five years struggling (often unsuccessfully) to write up results that I’d proven long before, and that I could easily explain in half an hour to anyone who asked me, and that only a tiny group of experts would ever need to see the details of, and that…

God, those years. They went by so quickly. They should’ve been the best years of my life. I should’ve been saving all of my scarce brain cycles for solving big, meaty problems. And in the meantime, I should’ve been learning how to salsa dance, and exploring San Francisco, and giving talks for elementary school kids, and having dumb affairs that I’d later regret. Instead, what was I doing? Responding to referee reports for this and this and this and this and this.

In an infamous guest post on Lance Fortnow’s blog, I advocated a rather different philosophy. I realize now that I was wrong, and I apologize to any colleagues who were hurt by what I said there. For I now understand that the true time-sucker is neither sailing nor surfing nor clubbing. These things take up only a finite number of hours in any case; once they’re done, they’re done. No, the true enemy of scientific productivity is having to write everything up in such a goddamned painstaking way.

I’ll estimate that I spend at least two months on writing for every week on research. I write, and rewrite, and rewrite. Then I compress to 10 pages for the STOC/FOCS/CCC abstract. Then I revise again for the camera-ready version. Then I decompress the paper for the journal version. Then I improve the results, and end up rewriting the entire paper to incorporate the improvements (which takes much more time than it would to just write up the improved results from scratch). Then, after several years, I get back the referee reports, which (for sound and justifiable reasons, of course) tell me to change all my notation, and redo the proofs of Theorems 6 through 12, and identify exactly which result I’m invoking from [GGLZ94], and make everything more detailed and rigorous. But by this point I’ve forgotten the results and have to re-learn them. And all this for a paper that maybe five people will ever read.

Let’s try some thought experiments. Steve Cook never bothered to write up a journal version of this STOC abstract. Suppose he did; what of it? After he wrote On Computable Numbers in 1936, Turing wrote an erratum in 1937, correcting a few bugs. Did you even know that? Do you care? Would Turing’s place in history be any different had he left the bugs unfixed?

So what’s the solution? Personally, my hope is that the Internet will eventually make not only traditional print journals obsolete (as it already has in some fields), but traditional papers as well. Instead we’ll have permanently-archived “interactive proofs”: discussions that look, more than anything else, like the emails exchanged between coauthors before they start writing up the paper.

“I think I can prove X like so.”
“But how do you handle Y and Z?”
“Well, what about W?”

Assuming the participants are serious researchers, I believe that the ”limit” of such a discussion is every bit as reliable as the paper itself. After all, if absolute rigor is the goal, then you shouldn’t believe the paper either. You should insist that everything be formalized in ZF set theory — and even then, how would you know that what was proved corresponded to the informal statement?

(To be clear, I’m not advocating some sort of woo-woo philosophy of mathematics. I don’t have a philosophy of mathematics — or if I do, then it’s naïve Platonism. All I’m advocating is that we consistently adopt the same standards of convincingness that we already adopt when arguing in front of a blackboard. I leave as an open problem how all of this applies to the “softer” sciences, like biology or string theory.)

But until the post-paper world I’m championing becomes a reality, what should you do? Here’s my advice: write the most informal, sloppy, essayistic, stream-of-consciousness, conversational papers you can possibly get away with. Write as if you were firing off an email to a skeptical but impatient friend. I promise to do my part by reviewing such papers leniently (at least in terms of the presentation), and no longer demanding pointless revisions.

Follow

Gosh, people found this blog quickly! Let’s see if I can make it the #1 shtetl on Google, just as Dave Bacon has installed himself as the #1 Pontiff.

Follow

You’re probably wondering about the title. I chose it because (1) I was pretty sure it wouldn’t be taken already, and (2) it evokes what I’ve come to see as the defining problem of my life.

A couple years ago, when I was still at Berkeley, some fellow grad students and I were discussing who we would have been had we lived 700 years ago. Would we have been farmers and bricklayers, like pretty much everyone else? Or kings and duchesses, as the people who go to Renaissance Fairs all apparently were? More interestingly, would we have found some way to capitalize on our bizarre “gifts” — say, by developing a new field-ploughing algorithm that was provably within a 1+ε factor of optimal? (That we would still have our nerdy gifts wasn’t open to question — for if we didn’t, then we would no longer be ourselves.)

“I would’ve been the chief rabbi of my shtetl,” I said. “All day long, I’d debate questions like how much restitution you’d have to pay if your ox gored your neighbor’s sheep. And for this, I’d get an arranged marriage with the most beautiful girl in town.”

Someone interjected that I shouldn’t sentimentalize too much: “After all, Scott, you wouldn’t have had any complexity theory! Or quantum computing!”

“That’s true,” I confessed. “So, you know — you win some, you lose some.”

Complexity and quantum computing will indeed play a role on Shtetl-Optimized, as will politics, economics, history, free will, global warming, The Simpsons — the whole megillah, or at least the sections I’ve skimmed. But the overarching theme will be “how to survive as a penguin in the desert, or a camel in Antarctica, or a caveman in — well, anyway, as someone who feels himself exquisitely adapted to an environment utterly unlike the one into which he was born.” If you’ve ever felt that way– or even if you’d just enjoy the spectacle of someone groping his way out of the nebbish-bin of history — I hope you find it enlightening.

Follow