A trivial post
Why do academics feel the need to stuff their papers with “nontrivial” results? After all, if a paper is remembered decades after it was written, it’s almost always for a simple core idea — not for the extensions and applications that fill 90% of the paper’s bulk.
The nontriviality virus can infect even the greats: think of Leonid Levin’s famous paper on universal search. According to legend, the reason Levin was scooped by Cook and Karp is that he spent a year trying to prove Graph Isomorphism was NP-complete! You see, that would’ve been a deep, publication-worthy result, unlike the “obvious” fact that there exist natural NP-complete problems.
Here’s a more recent example. In my opinion, this 43-pager by Barak et al. is one of the sweetest computer science papers of the past decade. But what makes it so sweet is a two-sentence insight (my wording):
There’s no generic, foolproof way to “obfuscate” a computer program. For even if a program looked hopelessly unreadable, you could always feed it its own code as input, which is one thing you couldn’t do if all you had was a “black box” with the same input/output behavior as the program in question.
So why did the authors go on for 43 more pages?
One possibility was suggested to me by Robin Hanson, an economist at George Mason who spews interesting ideas out of his nose and ears. Depending on your prejudices, you might see Robin as either a visionary futurist or a walking reductio ad absurdum of mainstream economic theory. Either way, his web page will surprise and provoke you.
When I talked with Robin in August, he speculated that nontrivial results function mainly as “certificates of smartness”: that is, expensive, difficult-to-fake evidence that the author(s) of a paper are smart enough that their simple core idea is likely to be worth taking seriously. Without these certificates, the theory goes, we academics would be deluged by too many promising ideas to entertain them all — since even if the ideas are simple, it usually isn’t simple to ascertain their worth.
Note that this theory differs from a more standard complaint, that academics fill their papers with nontrivial results for the sole purpose of getting them published. On Robin’s account, nontrivial results actually are useful to readers, just not in the way the paper advertises. Think of the author as a groom, the reader as a bride, and the nontrivial result as a wedding ring. The bride doesn’t care about the actual ring, but she does care that the groom was rich and devoted enough to buy one.
One prediction of Robin’s theory would be that, once you’ve established your smartness within the community, you should be able to get papers published even if they contain only simple observations. Another prediction would be that, if you’re very smart but emotionally attached to a simple idea, you should be able to buy exposure for your idea by encrusting it with nontrivialities. (As Robin remarked to me, everything in social science is either obvious or false; the only question is which.)
I don’t have anything deeper to say about Robin’s theory, but I’m enjoying the freedom to blog about it anyway.
Follow
Comment #1 October 19th, 2005 at 12:09 am
(NB Titles for blog entries would still be most useful.)
I don’t completely agree with the premise of this question. Many authors work too hard, or wait too long, to add “non-trivial” results to their papers. (And I may be one of them.) But many other authors make the opposite mistake: they make papers out of busywork that lacks any good idea. I think that the first kind of mistake is at least more honorable than the second. And I also think that in our information-choked age, the second problem is more widespread and more insiduous too.
After all, do you think that people who set out to maximize their “H-index” or equivalent wait until they have done something hard?
But I agree that if you have an idea, you should carefully weigh its merits, even if it is a simple idea. If it is a good idea, you should not wait to top it with something else hard or complicated.
The catch is that weighing the merits of a new idea is not so easy. One honorable way to test it, even if it is sometimes a mistake, is to push it further with that non-trivial extra. So I think that Levin’s mistake (as you describe it) is excusable; it was not an unmotivated goof.
On the other hand, Karp especially has had the brilliant, deft touch of identifying important ideas that are simple or only medium-complicated.
Comment #2 October 19th, 2005 at 4:09 am
Different papers are “stuffed” with results for different reasons.
For example, you’re right that the insight of the obfuscation paper can be summarized in two sentences, but at least for me, I needed the 43 pages to validate that insight. This is especially true for papers (such as this one) showing a negative result. The natural response to a negative result is to try to see if we can’t bypass it by changing some of the underlying assumptions. Since the underlying assumptions can be changed in many natural ways, the result is a 43 page paper.
There are other reasons I sometimes want to add more results to a paper. Suppose that there is some nice lemma that you came up with while working on the paper, which at the end turned out to be related but not strictly needed to the main result. My tendency would be to add it to the paper anyway. First, for my sake, if I’ll ever need that lemma I’ll have it in a written, verified and polished form. Also others may find it useful. Sometime they may even find this “side lemma” more useful than the main result.
–Boaz
Comment #3 October 19th, 2005 at 8:45 am
I agree with greg k. that many people try to cover up the fact that their paper doesn’t really have any new ideas. This can be called “result obsfucation”.
Comment #4 October 19th, 2005 at 8:57 am
You might think once you’ve established that you love your wife, you wouldn’t need to reassure her everyday by telling her that you love her. But of course in reality the day you stop telling her is the day she starts to fear you have stopped loving her. Similarly, I would not necessarily predict that you could publish simple ideas by themselves once you have established yourself – if people fear you may have “lost it” since your prime, you may need to keep proving yourself.
Comment #5 October 19th, 2005 at 9:14 am
In my conversation with Scott, I was actually emphasizing the idea that the main customers of academia, such as students and research patrons, primarily care about being able to associate with credentialed-as-impressive people. So all that busy-work, allowing academics to be credentialed as up to the task, is the main product for such customers. Those customers, and most academics themselves, don’t really care much about intellectual progress – that is a side effect. So rates of intellectual progress can vary greatly depending on rather random circumstances.
Comment #6 October 19th, 2005 at 10:31 am
If anything, I have more credentials than most of my students (calculus students) would want. I think that they liked me better when I was a graduate student.
Although it is true that student evaluations are not the only way to appraise teaching. If it were, research universities would have a hard time recruiting undergraduates.
Even so, I think that the real issue is the research itself, and not either student esteem or self-esteem. We are all frustrated by having to write gory proofs in papers, and by having to add macho theorems to the sweet aphorisms in the abstract. But in both cases, you have to earn the right to skip the hard part. If you are sure that someone can fill in the hard part, from any combination of divine inspiration and your own private sweat, then you have earned it. Otherwise skipping the hard part is either foolish or shallow.
Comment #7 October 19th, 2005 at 11:43 am
Of course Boaz is correct. Cryptography especially is full of anecodotes about how “obvious” (but still very clever!) statements were completely wrong, or required significantly more proof before they were actually proved to be true.
Comment #8 October 19th, 2005 at 12:13 pm
Contrary to what Greg says, one of the advantages of the h-index is that it discards busy work and focuses on papers which have had an impact. As your h-index grows this effect increases even further: your busy work counts for less.
One disadvantage of the h-index is that relevance of a result is not always measured by number of citations.
For example, what I believe to be my best result (settling a long standing conjecture on the size of a search engine index) has very few citations, since the paper pretty much closes the problem.
But overall it is definitely better than the simple number-of-papers count which is sometimes used by unenlightened tenure committees.
Alex Lopez-Ortiz