Open Peer Review: a Model & an Invitation (2019 update)

This is a 2019 update of a post originally published in 2005 on The Imaginary Journal of Poetic Economics; the original is republished here. This version reflects experience with open peer review (mine and that of others), further reflection, and research conducted since 2005.

These are some ideas for open peer review that can be used today in experiments that may be helpful to shape future systemic approaches. The overall goal is to facilitate open research by opening up preprints, increase transparency in the peer review process, and to allow peer reviewers to take credit for their work. Interested authors and/or reviewers can experiment with this approach today. For example, an author can post a preprint in a repository, seek volunteer reviewers through a listserv or other social media service for a relevant scholarly community and/or ask a colleague to serve as an editor to coordinate the review process and/or serve as a contact for blind reviews.

Examples and links:

Gibney (2016) wrote an article for Nature on peer review overlay journals built on arXiv that includes links to the journals.

A copy of my peer review of a recent article can be found in the University of Ottawa Institutional Repository here: https://ruor.uottawa.ca/handle/10393/39053 In this way my peer review constitutes a scholarly work with a stable URL that I include in my online CV.

See the Open Access Tracking Project tag oa.open_peer_review for 32 items as of July 15, some posted in the past week.

This model would be compatible with, but does not depend on, peer review overlay journals, featuring an overlay of peer review on articles submitted to and archived in institutional and/or subject repositories, the method recommended in 2009 for the UK for transition to open access in the medium to long term by Houghton et al. (2009) as the most transformative and most cost-effective approach.

The idea of open peer review is not new. While this post will not include a full review of related literature, as one example, Stevan Harnad talks about one approach to open peer review as early as 1996, in Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals.

The goals of this model are:

transparent peer review: unlike blind peer review, readers can see the peer review process in action. Rather than accepting an assessment of certification based on a closed system, readers can judge the peer review process per se, for themselves. This model could accomodate a combination of open and blind peer review – that is, a peer reviewer could publish a signed peer review, or provide comments confidentially, depending on the preferences of authors or the discretion of editors. As an example of the latter, when reviewing opinion pieces in an emotionally heated area, some blind review might be seen as preferable to open peer review.

increased scholarly literacy: it is assumed that a transparent peer review process will facilitate science literacy teaching, as more people will be able to see the peer review process in action

better peer review: exposing the peer review process per se will allow for thoughtful reflection on peer review per se, and facilitate research. This will allow for the development of better and more efficient peer review.

peer-reviewer credit: peer review is an important task, which a great many academics undertake on a voluntary basis. A portfolio of signed peer reviews can be added to the author’s c.v. The best peer-reviewers, those who are thorough, considerate, and respond quickly, can be recognised for their work.

facilitate and recognise author controlled peer review: There are advantages and disadvantages to author-controlled peer review, where the author takes responsibility to seek out peer reviewers. While this is not presently recognised as peer review, it is widely practiced. In the author’s view, an article which has been peer reviewed and edited accordingly prior to submission for publication, is likely to be a better article. Authors who seek out comments from colleagues, and peer reviewers who are sought out by authors, are both demonstrating an openness to collaboration and willingness to listen to critique – both important elements in conducting scholarly research. Author controlled peer review could be used to supplement editor-coordinated peer review (a pre-peer-reviewed article might need only one outside peer reviewer, for example, while an unreviewed work might need two or three).

In some cases, author controlled peer review could be an alternative to editor-coordinated peer review. It would be desirable to develop a set of criteria outlining the optimum for peer review (peer reviewer meets certain criteria, is not a former student, teacher, co-researcher or co-author, at least one peer reviewer from a different cultural background – more important in social than hard sciences – and so forth). Authors should explain whether and how they have met these criteria; this could be accomplished by an automated list, where the relevant criteria are checked off. Some of this could be be automated, as well – for example, a database of the author’s works will reveal former co-authors, and automated comparison of the c.v.’s of author and peer reviewer will reveal common affiliations.

Comments on this blogpost or via e-mail are welcome.

Last updated July 15, 2019.

References

Gibney, E. (2016). Open journals that piggyback on arXiv gather momentum. Nature 530: 7558. Retrieved July 15, 2019 from https://www.nature.com/news/open-journals-that-piggyback-on-arxiv-gather-momentum-1.19102

J. Houghton, B. Rasmussen, P. Sheehan, C. Oppenheim, A. Morris, C. Creaser, H. Greenwood, M. Summers, and A. Gourlay, 2009. “Economics implications of alternative scholarly publishing models: Exploring the costs and benefit” (27 January). Retrieved July 11, 2019 from:

https://pdfs.semanticscholar.org/45dd/cb9ebb9c8505a4ac86718734dda3311f91d8.pdf

See also
Behavioral and Brain Sciences (BBS) Editorial 1978 on Open Peer Commentary Thanks to Stevan Harnad.

Open Peer Review: A Model & An Invitation

In preparation for some current work in open peer review, this is a re-publication of my August 15, 2005 post on The Imaginary Journal of Poetic Economics (IJPE), drawing from and building on Harnad’s 1996 work in this area. IJPE is live but some readers (including myself while at my office) are reporting that they are not able to connect. Other links have not been tested.

Open Peer Review: A Model & An Invitation

This is one model for an open peer review system. The idea is to automate a great deal of the coordination of peer review, make much of it transparent, and allow peer-reviewers to take credit for their work. This model could fit well with either an institutional repository / peer review overlay approach, or a traditional journal approach for either OA or non-OA journals, or any combination thereof. Readers are welcome to comment, peer-review, and/or experiment with software approaches based on this model [reference to a copyright approach no longer used deleted].

The idea of open peer review is not new. While this post will not include a full review of related literature, as one example, Stevan Harnad talks about one approach to open peer review as early as 1996, in Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals.

The goals of this model are:

transparent peer review: unlike blind peer review, readers can see the peer review process in action. Rather than accepting an assessment of certification based on a closed system, readers can judge the peer review process per se, for themselves. This model could accomodate a combination of open and blind peer review – that is, a peer reviewer could publish a signed peer review, or provide comments confidentially, depending on the preferences of authors or the discretion of editors. As an example of the latter, when reviewing opinion pieces in an emotionally heated area, some blind review might be seen as preferable to open peer review.
increased science literacy: it is assumed that a transparent peer review process will facilitate science literacy teaching, as more people will be able to see the peer review process in action
better peer review: exposing the peer review process per se will allow for thoughtful reflection on peer review per se, and facilitate research. This will allow for the development of better and more efficient peer review.
peer-reviewer credit: peer review is an important task, which a great many academics undertake on a voluntary basis. A portfolio of signed peer reviews can be added to the author’s c.v. The best peer-reviewers, those who are thorough, considerate, and respond quickly, can be recognised for their work.
automate coordination of peer review: it should be possible to establish databases of peer reviewers, most likely distributed databases with central harvesting of key metadata (similar to institutional repositories & OAI), interoperable with other relevant software programs such as publishing software and calendaring systems, to automate much of the coordination of peer review.
peer review improvements through automation: the efficiencies of automation may make it possible to enhance peer review in ways that are not feasible with a system relying largely on one-on-one contact between editor and peer review. For example, there are many good reasons why it might be desirable to seek out an international peer review panel. An automated system would make it possible to easily identify experts in far-away countries, that the editor is unlikely to know personally. It is also possible to think about peer reviewers checking bits of an article, rather than the whole thing. That is, one paragraph of an article may refer to a completely separate area of expertise from the speciality of the author and main peer reviewers; there could be opportunities to ask a specialist to check just the one paragraph, rather than the whole article.
facilitate and recognise author controlled peer review: There are advantages and disadvantages to author-controlled peer review, where the author takes responsibility to seek out peer reviewers. While this is not presently recognised as peer review, it is widely practiced. In the author’s view, an article which has been peer reviewed and edited accordingly prior to submission for publication, is likely to be a better article. Authors who seek out comments from colleagues, and peer reviewers who are sought out by authors, are both demonstrating an openness to collaboration and willingness to listen to critique – both important elements in conducting scholarly research. Author controlled peer review could be used to supplement editor-coordinated peer review (a pre-peer-reviewed article might need only one outside peer reviewer, for example, while an unreviewed work might need two or three).

In some cases, author controlled peer review could be an alternative to editor-coordinated peer review. It would be desirable to develop a set of criteria outlining the optimum for peer review (peer reviewer meets certain criteria, is not a former student, teacher, co-researcher or co-author, at least one peer reviewer from a different cultural background – more important in social than hard sciences – and so forth). Authors should explain whether and how they have met these criteria; this could be accomplished by an automated list, where the relevant criteria are checked off. Some of this could be be automated, as well – for example, a database of the author’s works will reveal former co-authors, and automated comparison of the c.v.’s of author and peer reviewer will reveal common affiliations.

The model

Peer Reviewer Profiles
An academic who is willing to participate in peer review process creates a profile, which could be stored in the institutional repository. Elements of the profile could include:

  • author name
  • affiliation
  • title / position
  • areas of expertise (ideal might be using a standard list)
  • qualifying notes to each area of expertise – e.g., research specialist, practitioner expert
  • links to author’s own works
  • links to samples of work – open, signed peer reviews
  • comments from authors and/or editors
  • comments from recognised experts on the peer-reviewer’s expertise / ability to peer review in a particular area
  • author’s availability – time and number of peer-review requests the author is willing to accept at any given time.
    The time element could potentially be integrated with calendaring systems, e.g. no or fewer requests at particular times
  • author preferences for peer review – e.g. open access and/or fully green journals preferred, professional researchers only, researchers from developing countries welcome, students welcome (in limited numbers, perhaps?)
  • mutuality – in areas of controversy, authors might elect to publish critical reviews from peers with different perspectives, on the condition that their peer mutually publishes the author’s own peer review. This could provide readers with a good service, in alerting them to the existence of alternate viewpoints.

At the Institutional Repository

  • hosting or linking to author profiles and peer review
  • flexibility to accomodate clusters of versions. For example, lead readers first to the final peer-reviewed version, when available, but also make it easy for readers to find the original draft and peer reviewers’ comments.

Publishing software

  • links to author profiles
  • links to peer reviews
  • means of matching available peer reviewers with authors, editors, journals, or other certifying bodies

Comments or peer reviews can be posted as comments or sent to my contact e-mail found here. Any comments or reviews may be incorporated in future versions of this model. Please indicate if you are willing to allow your comments or review to be posted on this blog.

Comments

Peter Suber, August 18, 2005:

Note: Peter wants me to make clear that he does *not* believe that OA depends on peer-review reform, that OA has to wait for peer-review reform, or that OA is valuable primarily for its contribution to peer-review reform. OA is compatible with every kind of peer review and we should pursue it regardless of our position on peer review. (I completely agree, by the way!)

“Just for the record, I believe that peer-review definitely needs improvement and that many promising reforms have exciting synergies with OA. One of my pet ideas (which I wrote about more in the early days than recently) is retroactive peer review. Put the preprint in an OA repository as soon as it’s ready, then apply for review from a journal or free-floating editorial board. If approved, with or without revision, the approved version is also put in the repository with a citation and metadata showing its approved status. So far, this is just an overlay journal. What’s most exciting is the prospect of multiple editorial boards reviewing the same work, say, from different methodological or disciplinary perspectives, with the possibility of each giving (or withholding) its approval, creating something like a market in endorsements and tools that can search and sort by endorsement.”

E-LIS already has many of the components needed
by: Heather Morrison

E-LIS, the open archive for Library and Information Science, already has many of the components that would be needed for an open peer review system. One can already add comments to articles already in the archive – a reviewer could indicate if a comment is intended as a peer review, and link to a Peer Reviewer Profile. All that is needed is some editorial oversight, and communication with the author, and we’re almost there!

An illustration
An illustration of open peer review in action can be found in my Dramatic Growth of Open Access: Revised Update. This illustrates how an update to a peer-reviewed article can be improved, based on helpful constructive criticism on invitation from a friend.

Head and Neck Medicine
Head and Neck Medicine, a new Open Access Journal from BioMedCentral, is planning to follow an open peer review approach. Thanks to Open Access News, Aug. 30.

See also
Behavioral and Brain Sciences (BBS) Editorial 1978 on Open Peer Commentary Thanks to Stevan Harnad.

Last updated September 26, 2005. Minor modifications July 5, 2019.

Cite as:

Morrison, H. (2019). Open Peer Review: A Model & An Invitation. Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://sustainingknowledgecommons.org/2019/07/15/open-peer-review-a-model-an-invitation/

 

Ottawa open access workshop April 1

If you are in Ottawa on Saturday, April 1, please join the open access class for an all-day workshop on OA. Registration is free, but appreciated if you plan to join us for lunch. Details and a link to register can be found here: https://oa20171avril.wordpress.com/2017/03/18/all-day-open-access-workshop-at-university-of-ottawa/

Thanks to students of ISI 6300, a new master’s-level special topics course on open access, for organizing the event. Sustaining the Knowledge Commons is one of the sponsors.

Citation: cite the workshop link and original presenter (where applicable), not this blogpost.

Editorial: Open access, copyright and licensing: basics for open access publishers

by Heather Morrison & Lisa Desautels

Just published (February 2016) in the open access Journal of Orthopaedic Case Reports at the invitation of Editor-In-Chief Dr. Ashok Shyam: Editorial: open access, copyright and licensing: basics for open access publishers. Journal of Orthopaedic Case Reports 6:1 p. 1-2. DOI: 10.13107/jocr.2250-0685.360

Cite the original article (not this blogpost) as:

Morrison, H., & Desautels, L. (2016). Editorial – Open access, copyright and licensing: Basics for open access publishers. Journal of Orthopaedic Case Reports. Retrieved from http://www.jocr.co.in/wp/2016/01/02/2250-0685-360-fulltext/

Subject classification of DOAJ journals – data and documentation

Data for the subject classification of DOAJ journals is now available in the OA APC dataverse:

Morrison, Heather; Villamizar, César; Mondésir, Guinsly; Calvé-Genest, Alexis, 2019, “OA APC – Subject Analysis – Statistic Frequency & Coding”, https://doi.org/10.5683/SP2/4WKTZF, Scholars Portal Dataverse, V1

Documentation of the dataset:

Documentation – Subject Classification – OA APC – V1.3

Presentation of results:

Morrison, H., Villamizar, C., Salhab, J., & Calvé-Genest, A. (2015). Open access APC subject, content and impact factor correlational study. Presented at the Canadian Association of Information Studies annual conference, Ottawa, ON, Canada. Retrieved from https://ruor.uottawa.ca/handle/10393/32440

Cite as:

Villamizar, C., Mondésir, G., Calvé-Genest, A., & Morrison, Heather. (2015). Edit Post ‹ Sustaining the Knowledge Commons / Soutenir les savoirs communs. Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://wordpress.com/post/sustainingknowledgecommons.org/334