Made out of people

Does Bill Tozier have it right?

Bill Tozier, eBay observer, Erdos-seller, and diverse thinker, wants to create a new scientific community. One in which papers can be openly presented, hashed to pieces, and the best of them can be extracted and the strongest ideas revised. In the article linked, he discusses how to deal with the various crackpots and loonies who would inevitably become involved in the space.

The short summary is that a combination of rules and interest by the contributors. The contributors who have failed to take into account a substrantial body of literature should be directed to it (where they can learn how to use vocabulary well and understand what's already been done); the space should have rules about standards of typography, language, and presentation (to avoid the hand-scrawled diagrams with random capitalization).

These seem like perfectly reasonable standards for the sort of online, multi-edited journal that has been proposed for years. Indeed, arXiv.org seems to be slowly evolving into something like this: it's changed from pre-prints to "e-Prints"; some articles are printed in it that appear nowhere else. (Why they appear nowhere else is a matter of specuilation left to the reader).

My concern with Tozier's system is the critical mass problem. I suspect that mad geniuses will be delighted to use it, once they are convinced that the inability to use the blink tag doesn't actually damage their content. (That is, assuming that the contingent wants to be published in well-read sources.)

But what about everybody else? Will there be a large enough population to wade through the chaff and find the wheat? When reviewing for a major conference this year, I found a paper that flagrantly disregarded twenty years of research: not that it drew opposite conclusions, but that it simply failed to note that anyone had thought about the topic before, and wrote itself as a sort of prologomena to the field. Just getting one of them made me rather irritated at the authors; what would have been the effect of dozens of those?

For me, I would have stopped reading. And gone and swam in a pool that wasn't so polluted.

How do you ensure that it's worth the commentator's time to participate in a community? How do you make sure the community is big enough to provide value? CSCW research has lots of studies of communities that failed, didn't really take off, or sputtered. While Preece's book tries to engage this issue, I don't think there are yet canonical bullet points for "making sure your community gets adopted."

I'm curious to see where Bill's work goes with this.