Fleep's Deep Thoughts Explore the Metaverse

In the last year, I’ve had my first “real” papers published. I’ve been surprised at the long lead times, and the even longer review processes, and even looonger publishing dates before these things see the light of day. In a time when I can publish the same material myself with the click of a button, and get nearly instant feedback from colleagues, and when technology, techniques, and tools seem to be changing so rapidly that what was news a year or two ago seems woefully out of date already, the publishing process seems.. convoluted and a little bit absurd.

Image via Wikipedia

I understand, of course, the importance of peer review, and the need for data to be verified, experiments repeated, and findings confirmed or rejected. I understand the long standing traditions behind academic publishing, and agree that there is a real need for quality control and qualified, expert evaluation of information. But is this really the best we can do in the digital age? I don’t think so.

I just read a paper today that talks about this, and I think it raises some interesting points.

Kelty, Christopher M., C. Sidney Burrus, Richard G. Barniuk. (2008) “Peer Review Anew: Three Principles and a Case Study in Postpublication Quality Assurance“. IEEE.

They identify 3 fallacies and 3 principles that should be applied to modern peer review, and from the article, I think this is really key:

..simply placing something on the internet is not the same as “publishing” it. [..] A key insight that governs all of these principles is that quality is not an intrinsic component of the content of a work but rather a feature of how that work is valuable to a specific community of users: its context of use.

This reminds me of Henry Jenkins’ talk from ELI earlier this year, What Wikipedia Can Teach Us About the New Media Literacies (it’s a good talk, worth watching the video archive). A given wikipedia article may NOT be high enough quality to serve as a reference for a chemist, but 9 times out of 10, it IS high enough quality to give basic information for the rest of us non-chemists. And it can serve as a useful tool for teaching students about how “knowledge” is created, disputed, transformed, and disseminated. Context matters.

And of course, when we talk about new digital media, we haven’t even gotten into how to do peer review of, say, learning objects in 3D worlds like Second Life. We’ve taken a stab at creating some criteria for the Second Life Education Community Conference 2008, but it’s a first iteration of something that I hope to see become much more refined, and evaluating the content without knowing the context makes the job that much more difficult.

This is a topic I’m just starting to explore, so if anyone has pointers or resources, I’d love to hear them.