Content by Cory Doctorow (top ebook reader .TXT) 📕

get behind.

Ford Prefect, a researcher for the Hitchhiker's Guide to the Galaxy, has been stationed on Earth for years, painstakingly compiling an authoritative, insightful entry on Terran geography, science and culture, excerpts from which appear throughout the H2G2 books. His entry improved upon the old one, which noted that Earth was, simply, "Harmless."

However, the Guide has limited space, and when Ford submits his entry to his editors, it is trimmed to fit:

"What? Harmless? Is that all it's got to say? Harmless! One
word!"

Ford shrugged. "Well, there are a hundred billion stars in the
Galaxy, and only a limited amount of space in the book's
microprocessors," he said, "and no one knew much about the Earth
of course."

"Well for God's sake I hope you managed to rectify that a bit."

"Oh yes, well I managed to transmit a new entry off to the editor.
He had to trim it a bit, but it's still an improvement."

"And what does it say now?" asked Arthur.

"Mostly harmless," admitted Ford with a slightly embarrassed
cough.

[fn: My lifestyle is as gypsy and fancy-free as the characters in H2G2, and as a result my copies of the Adams books are thousands of miles away in storages in other countries, and this essay was penned on public transit and cheap hotel rooms in Chile, Boston, London, Geneva, Brussels, Bergen, Geneva (again), Toronto, Edinburgh, and Helsinki. Luckily, I was able to download a dodgy, re-keyed version of the Adams books from a peer-to-peer network, which network I accessed via an open wireless network on a random street-corner in an anonymous city, a fact that I note here as testimony to the power of the Internet to do what the Guide does for Ford and Arthur: put all the information I need at my fingertips, wherever I am. However, these texts *are* a little on the dodgy side, as noted, so you might want to confirm these quotes before, say, uttering them before an Adams truefan.]

And there's the humor: every writer knows the pain of laboring over a piece for days, infusing it with diverse interesting factoids and insights, only to have it cut to ribbons by some distant editor (I once wrote thirty drafts of a 5,000-word article for an editor who ended up running it in three paragraphs as accompaniment for what he decided should be a photo essay with minimal verbiage.)

Since the dawn of the Internet, H2G2 geeks have taken it upon themselves to attempt to make a Guide on the Internet. Volunteers wrote and submitted essays on various subjects as would be likely to appear in a good encyclopedia, infusing them with equal measures of humor and thoughtfulness, and they were edited together by the collective effort of the contributors. These projects -- Everything2, H2G2 (which was overseen by Adams himself), and others -- are like a barn-raising in which a team of dedicated volunteers organize the labors of casual contributors, piecing together a free and open user-generated encyclopedia.

These encyclopedias have one up on Adams's Guide: they have no shortage of space on their "microprocessors" (the first volume of the Guide was clearly written before Adams became conversant with PCs!). The ability of humans to generate verbiage is far outstripped by the ability of technologists to generate low-cost, reliable storage to contain it. For example, Brewster Kahle's Internet Archive project (archive.org) has been making a copy of the Web -- the *whole* Web, give or take -- every couple of days since 1996. Using the Archive's Wayback Machine, you can now go and see what any page looked like on a given day.

The Archive doesn't even bother throwing away copies of pages that haven't changed since the last time they were scraped: with storage as cheap as it is -- and it is *very* cheap for the Archive, which runs the largest database in the history of the universe off of a collection of white-box commodity PCs stacked up on packing skids in the basement of a disused armory in San Francisco's Presidio -- there's no reason not to just keep them around. In fact, the Archive has just spawned two "mirror" Archives, one located under the rebuilt Library of Alexandria and the other in Amsterdam. [fn: Brewster Kahle says that he was nervous about keeping his only copy of the "repository of all human knowledge" on the San Andreas fault, but keeping your backups in a censorship-happy Amnesty International watchlist state and/or in a floodplain below sea level is probably not such a good idea either!]

So these systems did not see articles trimmed for lack of space; for on the Internet, the idea of "running out of space" is meaningless. But they *were* trimmed, by editorial cliques, and rewritten for clarity and style. Some entries were rejected as being too thin, while others were sent back to the author for extensive rewrites.

This traditional separation of editor and writer mirrors the creative process itself, in which authors are exhorted to concentrate on *either* composing *or* revising, but not both at the same time, for the application of the critical mind to the creative process strangles it. So you write, and then you edit. Even when you write for your own consumption, it seems you have to answer to an editor.

The early experimental days of the Internet saw much experimentation with alternatives to traditional editor/author divisions. Slashdot, a nerdy news-site of surpassing popularity [fn: Having a link to one's website posted to Slashdot will almost inevitably overwhelm your server with traffic, knocking all but the best-provisioned hosts offline within minutes; this is commonly referred to as "the Slashdot Effect."], has a baroque system for "community moderation" of the responses to the articles that are posted to its front pages. Readers, chosen at random, are given five "moderator points" that they can use to raise or lower the score of posts on the Slashdot message boards. Subsequent readers can filter their views of these boards to show only highly ranked posts. Other readers are randomly presented with posts and their rankings and are asked to rate the fairness of each moderator's moderation. Moderators who moderate fairly are given more opportunities to moderate; likewise message-board posters whose messages are consistently highly rated.

It is thought that this system rewards good "citizenship" on the Slashdot boards through checks and balances that reward good messages and fair editorial practices. And in the main, the Slashdot moderation system works [fn: as do variants on it, like the system in place at Kur5hin.org (pronounced "corrosion")]. If you dial your filter up to show you highly scored messages, you will generally get well-reasoned, or funny, or genuinely useful posts in your browser.

This community moderation scheme and ones like it have been heralded as a good alternative to traditional editorship. The importance of the Internet to "edit itself" is best understood in relation to the old shibboleth, "On the Internet, everyone is a slushreader." [fn: "Slush" is the term for generally execrable unsolicited manuscripts that fetch up in publishers' offices -- these are typically so bad that the most junior people on staff are drafted into reading (and, usually, rejecting) them]. When the Internet's radical transformative properties were first bandied about in publishing circles, many reassured themselves that even if printing's importance was de-emphasized, that good editors would always been needed, and doubly so online, where any mouth-breather with a modem could publish his words. Someone would need to separate the wheat from the chaff and help keep us from drowning in information.

One of the best-capitalized businesses in the history of the world, Yahoo!, went public on the strength of this notion, proposing to use an army of researchers to catalog every single page on the Web even as it was created, serving as a comprehensive guide to all human knowledge. Less than a decade later, Yahoo! is all but out of that business: the ability of the human race to generate new pages far outstrips Yahoo!'s ability to read, review, rank and categorize them.

Hence Slashdot, a system of distributed slushreading. Rather than professionalizing the editorship role, Slashdot invites contributors to identify good stuff when they see it, turning editorship into a reward for good behavior.

But as well as Slashdot works, it has this signal failing: nearly every conversation that takes place on Slashdot is shot through with discussion, griping and gaming *on the moderation system itself*. The core task of Slashdot has *become* editorship, not the putative subjects of Slashdot posts. The fact that the central task of Slashdot is to rate other Slashdotters creates a tenor of meanness in the discussion. Imagine if the subtext of every discussion you had in the real world was a kind of running, pedantic nitpickery in which every point was explicitly weighed and judged and commented upon. You'd be an unpleasant, unlikable jerk, the kind of person that is sometimes referred to as a "slashdork."

As radical as Yahoo!'s conceit was, Slashdot's was more radical. But as radical as Slashdot's is, it is still inherently conservative in that it presumes that editorship is necessary, and that it further requires human judgment and intervention.

Google's a lot more radical. Instead of editors, it has an algorithm. Not the kind of algorithm that dominated the early search engines like Altavista, in which laughably bad artificial intelligence engines attempted to automatically understand the content, context and value of every page on the Web so that a search for "Dog" would turn up the page more relevant to the query.

Google's algorithm is predicated on the idea that people are good at understanding things and computers are good at counting things. Google counts up all the links on the Web and affords more authority to those pages that have been linked to by the most other pages. The rationale is that if a page has been linked to by many web-authors, then they must have seen some merit in that page. This system works remarkably well -- so well that it's nearly inconceivable that any search-engine would order its rankings by any other means. What's more, it doesn't pervert the tenor of the discussions and pages that it catalogs by turning each one into a performance for a group of ranking peers. [fn: Or at least, it *didn't*. Today, dedicated web-writers, such as bloggers, are keenly aware of the way that Google will interpret their choices about linking and page-structure. One popular sport is "googlebombing," in which web-writers collude to link to a given page using a humorous keyword so that the page becomes the top result for that word -- which is why, for a time, the top result for "more evil than Satan" was Microsoft.com. Likewise, the practice of "blogspamming," in which unscrupulous spammers post links to their webpages in the message boards on various blogs, so that Google will be tricked into thinking that a wide variety