on August 10, 2010 by Simon Cockell, Daniel Swan and Phillip Lord in Process, Comments (3)

Knowledgeblog types and peer-review levels

This article describes the reviewing types of Knowledgeblog. It also highlights the
editorial and author responsibilities where appropriate.

Authors

Daniel Swan, Simon Cockell, Phillip Lord

Introduction

The purpose of a knowledgeblog is to provide a mechanism for formal publication, suitable for the web, and using modern web-based publication technology. The formality comes from two key principles: first, articles undergo some form of review; and, second, once articles are considered reviewed,undergo no further semantic changes. The first principle ensures that authors gain some form of feedback on their work, while readers gain an assurance that the work has reached an appropriate level of quality. The second principle ensures that once an article has been reviewed, statements cannot be withdrawn (although they can be later contradicted) but become part a stable part of the scientific record.

This article discusses three different types of kblog review process. Each type supports the two key principles, but achieves them in a different way. They differ primarily in the selection of reviewers and the degree to which an one or more authoritative individuals is involved in this process. A priori, we do not consider these different kblog types to be differentiated by the level of rigour or quality control they provide.

Knowledgeblog types

Three types of Knowledgeblog are recognised, “Public Review”, “Author Review” and “Editorial Review”. These can be briefly summarised by their requirements:

Public Review:

Posts are written by a registered author and there may be a period of public review for new articles

Author Review:

Authors actively solicit at least one peer reviewer for new articles

Editorial Review:

A named editor (or editorial board) will review new articles, or select a reviewer for new articles, independently of the author

Public Review

A “Public Review” Knowledgeblog is the most lightweight in terms of editorial involvement. However, it allows for unsolicited reviewing which means that it can reach a wide audience for potential opinion, unbiased by the authors or editors.

Scope

A kblog may select this type of review process for a number of reasons: articles may be required to be timely and rapid, and this overrides a more complex review period; articles may be short and discursive; articles may potentially impact many individuals and so require a wide set of reviewers without prior judgement as to their suitability.

There is a single stipulation that:

Posts are written by a registered author. There is a period of public review of new articles.

As such the Public Review Knowledgeblog requires an administrator (to approve accounts) and one or more authors to produce content. When an article is posted, it is posted with the intent that comments can be made on the published article, either in the comments section, or via the pingback mechanism in WordPress from another blog.

The definition of Public Review is broad, and can encompass a number of strategies, depending on the intention of the Knowledgeblog.

  1. Time-locked Public Review would allow authors to submit an article for public comment, and the article would remain “under review” for a specified length of time. After this time has passed the author can make it “reviewed”
  2. Authors may post content published directly as “reviewed” or “under review” as they see fit. In the first instance the intention should be that the final article is not to change post-review. In the second instance, the article may remain “under review” for as long as the author desires, and the audience understands that the article may change, until it has been reclassified.

There is no stipulation at this level that any article must have reviews in order to be considered a finished product.

Examples

This level of Knowledgeblog is used for the Taverna Knowledgeblog. Articles were written during a workshop and published immediately. Comments can be left on the articles for author feedback, however there is no assignment of peer reviewers either from the authors, or the editors of the Knowledgeblog.

Author Review

At this level, there is an expectation that the final published content has passed some measure of peer-review, either checking for factual accuracy or methodological soundness in the article. The reviewer is not selected by an editor or editorial board, but by the author themselves.

Scope

This level of Knowledgeblog is appropriate, for instance, where a body of information is being collected around a topic, but some oversight is required from a specified domain expert. Articles are published immediately, but remain clearly marked as ‘Under Review’ until a peer reviewer (selected by the author) has read the article, and returned feedback to the author. When this is done and the reviewers comments have been incorporated, the article can be promoted from “Under review” to “Reviewed” in the Knowledgeblog system. This version is considered final, but does not preclude post-publication peer review as in the “Public Review” type.

To summarise:

Registered authors actively solicit at least one peer reviewer for new articles, and incorporate reviewers comments into the final article.

Examples

This level of Knowledgeblog has been used for the Ontogenesis Knowledgeblog, a collaboratively written, online resource for descriptive, tutorial and explanatory material about building, using and maintaining ontologies. Articles are reviewed publicly (see the Peer Review category) and those that have associated reviews are promoted from the “Under Review” category to the “Reviewed” category.

Editorial Review

For an Editorial Review Knowledgeblog a clearly identified editor takes responsibility for assigning reviewers to new articles, or preselects a set of reviewers who pool expertise for reviewing Knowledgeblog articles.

The author may be involved in this process, but the editorial decision is final.

Scope

A more traditional, journal style Knowledgeblog is suited to this level of review. An editor or editorial board takes the responsibility of sourcing reviewers for new articles. As with the Author Review, articles can be published immediately and remain “Under Review” until the selected reviewer(s) have returned their comments to the author. The editor is responsible for promoting articles to “Reviewed” status, when satisfied the author has taken the reviewers comments into account. Again, post-publication peer review and comment is supported by the Knowledgeblog.

To summarise:

A named editor (or editorial board) will review new articles, or select a reviewer for new articles, independently of the author

Examples

Currently there are no Knowledgeblogs with this level of peer review, however the model is analagous to that of PLoS One and
Nature’s Scientific Reports but with the intention that reviewers comments are made public, and revisions to articles in the light of comments have a publicly exposed provenance trial in the form of accessible revision information

Implementation

Currently, within the knowledgeblog software, the implementation of the Knowledgeblog types comes through the use of the ‘Edit Flow’ wordpress plugin, which itself is fully customisable and will permit any number of permutations to suit both authors, reviewers and editors. However, the different types described here are intended to be generic enough to be implementable with any number of systems.

In Summary

This document presents an outline set of editorial and author requirements for three types of Knowledgeblog. These should be sufficiently broad to cater for most requirements for light-weight online publication. The levels are not prescriptive and may be extended, modified or adapted as a community sees fit. This document, however, gives a broad overview of different types, giving clarity to the overall review process.

Tags: , , , ,

3 Comments

  1. Ontogenesis: One Year On | Ontogenesis

    April 7, 2011 @ 2:39 pm

    […] Ontogenesis, we have addressed this in two ways; we have moved to a single, author directed reviewing system. Authors choose their own reviewer, and decide when they have addressed the […]

  2. How to Write a Review for Knowledge Blog | The Knowledgeblog Process

    May 30, 2011 @ 4:07 pm

    […] many journals, but is common in the process of many standards committees. Depending on the kblog level being used, the review may have been requested by the author of the article, the editor of the […]

  3. An Exercise in Irrelevance » Blog Archive » Permalink Semantics

    June 9, 2012 @ 7:52 pm

    […] So I changed it; for example, http://process.knowledgeblog.org/archives/19 became plain http://process.knowledgeblog.org/19. I don’t understand why, as WordPress seemed to maintain the links last time, but apparently […]

Leave a comment

Login