Open peer review: a preliminary review, an open offer, observations and discussion

This post links to a preliminary review of Debat & Babini’s preprint PlanS in Latin America: a precautionary note (citation details below) – in brief, Latin America has long been a leader and role model, and these authors have no peers; an open offer to conduct a full peer review (with conditions), and a link to a post highlighting my current perspective on open peer review and an invitation to participate in experimentation with, and discussion about, open peer review. A link to this post as an offer for a full open peer review will be sent to Debat, Babini, and the editor of PeerJ.

Open offer to conduct a full peer review (with conditions): if desired by the authors and the journal should the journal wish to accept my conditions, I offer to conduct a full peer review of this article under the following conditions:

  • My peer review would be open access but published under, and clearly marked as, All Rights Reserved Copyright, and will include a detailed explanation of this choice at the bottom of the peer review.
  • If the journal, PeerJ, wishes to publish the review, what is required:
    • An exception to the journal’s CC-BY policy
    • A mechanism for a “one-time-only” review, i.e. if I agree to review one article, this does not mean that I wish to join the PeerJ community as an author or receive further review requests
  • The authors and journal must commit to a particular version for the review, grant a reasonable time frame (minimum two weeks) for the review, and commit to reading and responding to the review. Rationale: it is not a good use of a reviewer’s time to review a version while the authors are already working on another version and/or if the work itself might be complete before the reivew.

The peer review itself becomes an item that I wish to retain and include in my CV, hence the official version from my perspective is included in my institutional repository. Following is a sample of a recent open peer review I did on a related topic:  https://ruor.uottawa.ca/hand
le/10393/39053

Observations and discussion

Please see my post Open Peer Review: a Model & an Invitation (2019 update) for current perspective and an invitation to participate in discussion and experimentation to further open peer review.

Debat & Babini article – citation, abstract and links

Humberto Debat, Instituto Nacional de Tecnología Agropecuaria (IPAVE-CIAP-INTA), Argentina

Dominique Babini, Consejo Latinoamericano de Ciencias Sociales (CLACSO), Argentina

Latin America has historically led a firm and rising Open Access movement and represents the worldwide region with larger adoption of Open Access practices. Argentina has recently expressed its commitment to join Plan S, an initiative from a European consortium of research funders oriented to mandate Open Access publishing of scientific outputs. Here we suggest that the potential adhesion of Argentina or other Latin American nations to Plan S, even in its recently revised version, ignores the reality and tradition of Latin American Open Access publishing, and has still to demonstrate that it will encourage at a regional and global level the advancement of non-commercial Open Access initiatives.

Access to full-text in English:

https://peerj.com/preprints/27834/

Access to full-text in Spanish:

https://zenodo.org/record/3332621#.XSekx-hKg2z

Open Peer Review: a Model & an Invitation (2019 update)

This is a 2019 update of a post originally published in 2005 on The Imaginary Journal of Poetic Economics; the original is republished here. This version reflects experience with open peer review (mine and that of others), further reflection, and research conducted since 2005.

These are some ideas for open peer review that can be used today in experiments that may be helpful to shape future systemic approaches. The overall goal is to facilitate open research by opening up preprints, increase transparency in the peer review process, and to allow peer reviewers to take credit for their work. Interested authors and/or reviewers can experiment with this approach today. For example, an author can post a preprint in a repository, seek volunteer reviewers through a listserv or other social media service for a relevant scholarly community and/or ask a colleague to serve as an editor to coordinate the review process and/or serve as a contact for blind reviews.

Examples and links:

Gibney (2016) wrote an article for Nature on peer review overlay journals built on arXiv that includes links to the journals.

A copy of my peer review of a recent article can be found in the University of Ottawa Institutional Repository here: https://ruor.uottawa.ca/handle/10393/39053 In this way my peer review constitutes a scholarly work with a stable URL that I include in my online CV.

See the Open Access Tracking Project tag oa.open_peer_review for 32 items as of July 15, some posted in the past week.

This model would be compatible with, but does not depend on, peer review overlay journals, featuring an overlay of peer review on articles submitted to and archived in institutional and/or subject repositories, the method recommended in 2009 for the UK for transition to open access in the medium to long term by Houghton et al. (2009) as the most transformative and most cost-effective approach.

The idea of open peer review is not new. While this post will not include a full review of related literature, as one example, Stevan Harnad talks about one approach to open peer review as early as 1996, in Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals.

The goals of this model are:

transparent peer review: unlike blind peer review, readers can see the peer review process in action. Rather than accepting an assessment of certification based on a closed system, readers can judge the peer review process per se, for themselves. This model could accomodate a combination of open and blind peer review – that is, a peer reviewer could publish a signed peer review, or provide comments confidentially, depending on the preferences of authors or the discretion of editors. As an example of the latter, when reviewing opinion pieces in an emotionally heated area, some blind review might be seen as preferable to open peer review.

increased scholarly literacy: it is assumed that a transparent peer review process will facilitate science literacy teaching, as more people will be able to see the peer review process in action

better peer review: exposing the peer review process per se will allow for thoughtful reflection on peer review per se, and facilitate research. This will allow for the development of better and more efficient peer review.

peer-reviewer credit: peer review is an important task, which a great many academics undertake on a voluntary basis. A portfolio of signed peer reviews can be added to the author’s c.v. The best peer-reviewers, those who are thorough, considerate, and respond quickly, can be recognised for their work.

facilitate and recognise author controlled peer review: There are advantages and disadvantages to author-controlled peer review, where the author takes responsibility to seek out peer reviewers. While this is not presently recognised as peer review, it is widely practiced. In the author’s view, an article which has been peer reviewed and edited accordingly prior to submission for publication, is likely to be a better article. Authors who seek out comments from colleagues, and peer reviewers who are sought out by authors, are both demonstrating an openness to collaboration and willingness to listen to critique – both important elements in conducting scholarly research. Author controlled peer review could be used to supplement editor-coordinated peer review (a pre-peer-reviewed article might need only one outside peer reviewer, for example, while an unreviewed work might need two or three).

In some cases, author controlled peer review could be an alternative to editor-coordinated peer review. It would be desirable to develop a set of criteria outlining the optimum for peer review (peer reviewer meets certain criteria, is not a former student, teacher, co-researcher or co-author, at least one peer reviewer from a different cultural background – more important in social than hard sciences – and so forth). Authors should explain whether and how they have met these criteria; this could be accomplished by an automated list, where the relevant criteria are checked off. Some of this could be be automated, as well – for example, a database of the author’s works will reveal former co-authors, and automated comparison of the c.v.’s of author and peer reviewer will reveal common affiliations.

Comments on this blogpost or via e-mail are welcome.

Last updated July 15, 2019.

References

Gibney, E. (2016). Open journals that piggyback on arXiv gather momentum. Nature 530: 7558. Retrieved July 15, 2019 from https://www.nature.com/news/open-journals-that-piggyback-on-arxiv-gather-momentum-1.19102

J. Houghton, B. Rasmussen, P. Sheehan, C. Oppenheim, A. Morris, C. Creaser, H. Greenwood, M. Summers, and A. Gourlay, 2009. “Economics implications of alternative scholarly publishing models: Exploring the costs and benefit” (27 January). Retrieved July 11, 2019 from:

https://pdfs.semanticscholar.org/45dd/cb9ebb9c8505a4ac86718734dda3311f91d8.pdf

See also
Behavioral and Brain Sciences (BBS) Editorial 1978 on Open Peer Commentary Thanks to Stevan Harnad.

Open peer review

MDPI’s journal is experimenting with open peer review. Reviewers can choose to make their reviews openly accessible. This recent article that I reviewed has just been published. To read my reviews, click on “review reports” – I am Reviewer 1. If you prefer to skip the details of work that was needed and subsequently done, skip to round 3 for the final review.

Following is the citation and abstract of the article and a portion of my final review that focuses on the work per se, and an update based on subsequent conversation with Frantsvåg. I will publish the copyright statement separately.

Frantsvåg, J.E.; Strømme, T.E. Few Open Access Journals Are Compliant with Plan S.  Publications 20197, 26. https://www.mdpi.com/2304-6775/7/2/26

Abstract

Much of the debate on Plan S seems to concentrate on how to make toll-access journals open access, taking for granted that existing open access journals are Plan S-compliant. We suspected this was not so and set out to explore this using Directory of Open Access Journals (DOAJ) metadata. We conclude that a large majority of open access journals are not Plan S-compliant, and that it is small publishers in the humanities and social sciences (HSS) not charging article processing charges (APC) that will face the largest challenge with becoming compliant. Plan S needs to give special considerations to smaller publishers and/or non-APC based journals.

Excerpt of final round of review:

An open peer review of “Few open access journals are Plan S compliant”: third and final round by Dr. Heather Morrison, Associate Professor, University of Ottawa School of Information Studies, and Principal Investigator, Sustaining the Knowledge Commons, a SSHRC Insight Project. Ó Dr. Heather Morrison, All Rights Reserved (explanation below).

This article presents important research and merits publication; conclusions are basically sound and recommendations appear timely and sensible.

One major substantive point of potential confusion remains. This confusion is evident is PlanS implementation guidance per se which states: “cOAlition S acknowledges that some publishers have established mirror journals with one part being subscription based and the other part being Open Access. Such journals are not compliant with Plan S unless they are a part of a transformative agreement since they de facto lead to charging for both access and publishing in the same way as a hybrid journal does. Funding for publishing in such journals will only be supported under a transformative agreement”. From: https://www.coalition-s.org/implementation/

It is not clear what PlanS is referring to here. The most common arrangement that seems to fit what is described here in my experience is journals that publish both in print (generally on a subscriptions basis) and online (on a fully open access basis, required for inclusion in DOAJ). A journal that is partially open access and partially subscription-based in its online form is a hybrid journal, contrary to PlanS advice.

This confusion is reflected in this article (Table 1 row G and results lines 461-467). Since this reflects the original, this should not be a barrier to publication.

Conflict of interest: although I am an open access advocate and my research focuses on transforming the underlying economics of scholarly publishing in order to sustain open access, I strongly disagree with the PlanS policy approach. In my expert opinion, all open access policy should require exclusively open access archiving. This is the best means to ensure preservation and ongoing open access, particularly in the region for which funders have responsibility. Market-oriented policy is likely to continue or exacerbate a problematic market that for decades has been described as inelastic at best.

Update based on e-mail with Frantsvåg: it appears that what PlanS means by mirror journals is an emerging phenomenon. Elsevier’s Journal of Hydrology X is an example. Following is the explanation of how this works from the Elsevier website. This appears to be an evolution in the hybrid journal model (some content open access, some not). This is an improvement in terms of access as it addresses criticism of the hybrid model on the basis that it is difficult to identify the open access content.

Journal of Hydrology X is the open access mirror journal of Journal of Hydrology

Journal of Hydrology X offers authors with high-quality research who want to publish in a gold open access journal the opportunity to make their work immediately, permanently, and freely accessible.

Journal of Hydrology and Journal of Hydrology X have the same aims and scope. A unified editorial team manages rigorous peer-review for both titles using the same submission system. The author’s choice of journal is blinded to referees, ensuring the editorial process is identical. For more information please refer to our FAQs for authors.