open peer review

OA Main 2019: Dataset, documentation and open peer review invitation

Posted by

0

Update Jan. 20, 2020

Thank you to those who provided comments. No changes were suggested for the dataset per se; the documentation has been revised to reflect the comments and the final version is now available here.

OA APC Main 2019 dataset documentation final

This is an invitation to participate in an open peer review of the OA APC Main 2019 dataset, its documentation and the value of research blogposts made possible through this project. While feedback on the OA APC project is appreciated at any time, the formal open peer review period is Dec. 1, 2019 – Jan. 15, 2020. My perspective is that open peer review is in an early phase where experimentation with different approaches could be useful to develop future best practices.

For this reason, reviewers are welcome to submit comments in the format or venue of their choice. Comments on this blogpost are the most convenient approach for the author. Signed comments are preferred. At the end of the formal open peer review period (Jan. 15), I will write a summary of the open peer review process, including all comments and responses where warranted. Comments to date and replies are posted here.

Links:

Questions to consider in the open peer review process:

The following are meant as guidelines only. Please feel free to focus on one or more specific questions that you find of interest and/or feel qualified to comment on, including questions not asked.

1. Importance and relevance of the research questions: the project’s research question and sub-questions (from the documentation) are as follows:

Research question

What trends can be observed in APCs over time? Subquestions:

    • Will competition emerge, or will an inelastic market transition or reappear?
    • Will the percentage of journals that are charging and non-charging remain the same or change?
    • Will fully OA journals continue to actively publish, cease, change to partially OA (hybrid) or non-OA journals?
    • What are the OA publishing and charging / non-charging trends and practices of particular publishers? (Note: results of individual research project done sustainingknowledgecommons.org)

Are any of the research questions important, relevant, or otherwise? Do you have any advice for the research team or potential downstream researchers using the dataset about research that you think will be important and relevant in future? Do you have any suggestions for further research?

2. Adequacy of the documentation: is the documentation of the dataset sufficient so that a downstream researcher could continue this research if desired and/or use a subset of the dataset for further research?

3. Limitations: Are the limitations of the dataset sufficiently well described? Is anything missing?

4. Alternative approaches: Are the alternative approaches sufficiently well described?

5. Errors in the dataset: Please note any errors found in the dataset (be specific).

6. Other: please provide feedback on any aspects of the dataset or its documentation not covered in the above questions.

Second update January 8: thanks to Heather Staines from MIT’s Knowledge Futures Group for initiating a conversation on open peer review processes via e-mail and agreeing to publish on the blog. This conversation (which prompted the initial procedural update) can be found here.

Procedural update January 8: there are diverse approaches to data documentation and open peer review. Peer review of an earlier version of this dataset and documentation was published in MDPI’s innovative journal Data, designed for this purpose (Morrison et al. 2017). Tools have been, and are being, developed to provide technical support for new, more open approaches to peer review such as hypothes.is, which facilitates online annotations. I see tremendous potential for open peer review, online annotations, and collaborative online writing. However, I see the process of transformation as in an early stage where experiments (like this one) and open discussion are more important than technical solutions. In other words, my choice of this approach – blogpost and open-ended invitation –  is deliberate. Discussion about open peer review process and potential is welcome, although it is a side-conversation to reviewing this dataset designed to facilitate study of longitudinal trends in article processing charges.

Cite as:

Morrison, H. (2019). OA Main 2019: Dataset, documentation and open peer review invitation. Sustaining the Knowledge Commons. Retrieved from https://sustainingknowledgecommons.org/2019/11/20/oa-main-2019-dataset-documentation-and-open-peer-review-invitation/

Reference
Morrison, H., Brutus, W., Dumais-Desrosiers, M., Kakou, T. L., Laprade, K., Mehri, S., … Wheatley, S. (2017). Open Access Article Processing Charges (OA APC) Longitudinal Study 2016 Dataset. Data, 2(2), 13. https://doi.org/10.3390/data2020013

 

Rev. Jan. 20, 2020

 

 

Peer review of Pubfair framework

Posted by

0

Peer review of Pubfair framework:

Ross-Hellauer, T.; Fecher, B.; Shearer, K.; Rodrigues, E. (2019). Pubfair: a framework for sustainable, distributed, open science publishing. White paper, version 1: Sept. 3, 2019. Confederation of Open Access Repositories (COAR).m Retrieved September 23, 2019 from https://www.coar-repositories.org/news-media/inviting-community-input-pubfair/

by: Dr. Heather Morrison, sustainingknowledgecommons.org

Highlights

“Science” is only one type of knowledge. There are nine faculties at the University of Ottawa; only one is named “science”, and this is typical at a large university. I strongly recommend replacing “science”, “scientists” and “open science” with more inclusive terminology such as “open scholarship” or “open knowledge”, “scholar” or “researcher” in the title and throughout the document. The Pubfair framework is an excellent beginning for a needed profound transformation in how scholars work together and disseminate research. This is the kind of approach most likely to achieve significant savings based on current spend on scholarly publishing, and these savings will be needed to support innovation in scholarly production and dissemination. My recommendation is to proceed with an iterative approach and an initial focus on helping scholarly communities with unmet needs for new forms of review and publishing, such as scholars who create and share datasets or tools using artificial intelligence, digital humanists, and scholarly bloggers. The specific needs for community input whether through review or collaboration in the planning process will vary by discipline and type of product. The work of defining needs and identifying potential solutions should be led by the scholarly community in consultation with repository managers. This is a reversal of the proposed leadership / consultation approach in the framework document. Finally, while I recommend an immediate start to this approach, my advice is to see this as a long-term radical transformation that will likely take decades to complete.

Details

Scholarship includes, but is not limited to, science. I suggest changing the title and wording throughout the document to more inclusive terminology such as “open scholarship” or “open knowledge”. To illustrate why this matters: the University of Ottawa (uO) is an indirect membership of the Confederation of Open Access Repositories (COAR) through our membership in the Canadian Association of Research Libraries (CARL).  uO has 9 faculties: Arts, Education, Engineering, Health Sciences, Law, Medicine, Sciences, Social Sciences, and the Telfer School of Management. If this framework for “open science publishing services” is to become a reality, is it intended to serve only one of our 9 faculties? This seems unlikely. This conflation of “science” with “knowledge” or “scholarship” is not unique to this group but reflects a broader trend in the open movements. The problem is much larger than mere semantics, it reflects a tendency in our society to devalue types of knowledge other than science. I argue that this is a danger to our collective knowledge of all types, including science. For example, would it be wise to practice science without ethics or logic? Ethics and logic are branches of philosophy. Readers of this review will probably agree that governments should base policy on scientific evidence. However, politics per se is not science, and achieving and maintaining a goal of basing policy on scientific evidence requires understanding of history, political science, society, and communications. I discussed this in a recent conference presentation called knowledge as a human right (Morrison, 2019a).

The Pubfair conceptual model of building a framework to transform scholarly publishing building on a distributed network of repositories is a timely initiative and worthy of support. This is the kind of approach most likely to facilitate transformative transition in terms of both technology and economics. Houghton et al. (2009) conducted the most comprehensive study of the potential for transformation for a single country (the UK), comparing potential costs of 3 models: gold open access publishing, green open access archiving, and a third more transformative approach involving a new publishing system building peer review on top of archives. The transformative approach was calculated as having the potential to substantially reduce costs and was seen as the most cost-effective approach. However, at the time the UK did not think the country was ready for this transformation and opted for a focus on gold and maintenance of a pre-existing green system.

Much has changed in the past 10 years. The number of open access repositories listed in the vetted OpenDOAR list has grown from 1,419 on June 20, 2009 to 4,150 on June 30, 2019. OpenDOAR lists repositories on 5 continents. The largest metasearch service for repositories and open access journals is the Bielefeld Academic Search Engine (BASE). From 2009 to 2019 (June 30 each year) BASE grew from 1,730 content providers and just over 25 million documents searched to 7,211 content providers and just under 150 million items searched. (Morrison, 2019b).

Scholarly works and their dissemination appear to be undergoing a period of rapid transformation in a way that has not been addressed by traditional approaches to evaluating scholarship, the traditional publishing business, or even the open access movement. In the digital humanities, scholars are creating collections of electronic works and developing innovative means of searching, processing, and displaying material. Scholars in a wide range of disciplines from art to engineering are using artificial intelligence to create new knowledge and practical tools. The disciplines themselves are undergoing change with new forms of scholarship often overlapping what used to be separate disciplines. A few researchers, like me, are publishing open research using blogs and likely other formats and sharing open data; more would likely follow suit if they could be confident that they would receive appropriate recognition for doing so when it comes time for tenure and promotion.

Given this context, for practical reasons I recommend an iterative approach, beginning with scholars who are interested in exploring alternatives and motivated to do so because current approaches do not meet their needs. There may be common themes across disciplines and types of research, but it will also be important to recognize differences based on the type of work and the nature of the communities that would need to configure or re-configure to accomplish this work.

Four examples:

  • Peer review of open datasets might focus on quality and completeness of data and documentation, and/or adherence to relevant standards, reference to related work, and/or importance of the dataset. Qualified peer reviewers need some expertise in quantitative data and the relevant domain; comments from those who might benefit from results would be helpful as well. The ideal outcome might involve collaboration in the process of developing datasets rather than peer review after publication, to avoid duplication or fully benefit from triangulation from different approaches to the same underlying problem.
  • Peer review of a digital humanities dataset and portal for users might focus on the quality of metadata, quality and comprehensiveness of content, usability and accessibility of the users’ portal, preservation planning, interoperability with relevant databases, or how licensing for re-use has been addressed. Here, different aspects of review involve different types of expertise, from content subject knowledge to user experience to electronic preservation. The benefits of collaboration in the planning process appear obvious.
  • Peer review of tools developed through AI to support the work of health professionals and/or to help patients monitor their own conditions may require triangulation using other methods to ensure accuracy of results, user experience analysis of the tools and/or periodic evaluation of the ongoing accuracy of AI assuming ongoing machine learning.
  • Peer review of scholarly and research blogs such as Retraction Watch, my scholarly blog The Imaginary Journal of Poetic Economics, and my research blog org might focus on the accuracy, originality, and/or importance of the contributions. In this case, review by subject experts is essential, and technical advice, which may be provided by different reviewers, is useful. Individual authors or groups of authors may or may not see collaboration in the planning process as useful. As a researcher-blogger, I can see situations where different types of blogs and authors would benefit from different types of review.

These examples are just a few of many possible new types of scholarship made possible by the digital environment. The optimal form of review and publishing such new types of works is, at present, unknown. This is another reason to seek an iterative approach and look for leadership within the communities of scholars pursuing these approaches. To understand what kind of review is most helpful for the community, it is necessary to understand in depth the nature of the research and/or creative works that are being developed.

Finally, this transition is a major cultural shift in how academics might work in future. It will take time to figure out the questions that will arise in the process, and more time to develop solutions. In summary, while I see this as a long-term transition, an approach along the lines of Pubfair is the right direction and steps should be taken to move in this direction as soon as possible.

Thank you for providing the opportunity to comment and best wishes for Pubfair.

About me

I write as a researcher focused on the transition of scholarly communication from the demand (subscriptions / purchase) to the supply side to support a global open access knowledge commons. My research project, funded by Canada’s Social Sciences and Humanities Research Council from 2014 – 2021, is called Sustaining the Knowledge Commons. My comments will be posted in the uO institutional repository and cross-posted to my open research blog, sustainingknowledgecommons.org.

References

Houghton, J.; Rasmussen, B.; Sheehan, P.; Oppenheim, C.; Morris, A.; Creaser, C.; Greenwood, H.; Summers, M. & Gourlay, A. (2009). Economics implications of alternative scholarly publishing models: Exploring the costs and benefit. A report to the Joint Information Systems Committee. UK. Retrieved July 11, 2019 from:

https://pdfs.semanticscholar.org/45dd/cb9ebb9c8505a4ac86718734dda3311f91d8.pdf

Morrison, H. (2019a). Knowledge as a human right. Presentation Jan. 30, 2019, University of Ottawa, cc-UNESCO Science as a human right series. http://hdl.handle.net/10393/38890

Morrison, H. (2019b). The Dramatic Growth of Open Access. June 30, 2019 full dataset. https://hdl.handle.net/10864/10660

September 24, 2019.

Cross-posted: cite as:

Morrison, H. (2019). Peer review of Pubfair framework. Sustaining the Knowledge Commons. https://sustainingknowledgecommons.org/2019/09/24/peer-review-of-pubfair-framework/

 

 

Open peer review: a preliminary review, an open offer, observations and discussion

Posted by

0

This post links to a preliminary review of Debat & Babini’s preprint PlanS in Latin America: a precautionary note (citation details below) – in brief, Latin America has long been a leader and role model, and these authors have no peers; an open offer to conduct a full peer review (with conditions), and a link to a post highlighting my current perspective on open peer review and an invitation to participate in experimentation with, and discussion about, open peer review. A link to this post as an offer for a full open peer review will be sent to Debat, Babini, and the editor of PeerJ.

Open offer to conduct a full peer review (with conditions): if desired by the authors and the journal should the journal wish to accept my conditions, I offer to conduct a full peer review of this article under the following conditions:

  • My peer review would be open access but published under, and clearly marked as, All Rights Reserved Copyright, and will include a detailed explanation of this choice at the bottom of the peer review.
  • If the journal, PeerJ, wishes to publish the review, what is required:
    • An exception to the journal’s CC-BY policy
    • A mechanism for a “one-time-only” review, i.e. if I agree to review one article, this does not mean that I wish to join the PeerJ community as an author or receive further review requests
  • The authors and journal must commit to a particular version for the review, grant a reasonable time frame (minimum two weeks) for the review, and commit to reading and responding to the review. Rationale: it is not a good use of a reviewer’s time to review a version while the authors are already working on another version and/or if the work itself might be complete before the reivew.

The peer review itself becomes an item that I wish to retain and include in my CV, hence the official version from my perspective is included in my institutional repository. Following is a sample of a recent open peer review I did on a related topic:  https://ruor.uottawa.ca/hand
le/10393/39053

Observations and discussion

Please see my post Open Peer Review: a Model & an Invitation (2019 update) for current perspective and an invitation to participate in discussion and experimentation to further open peer review.

Debat & Babini article – citation, abstract and links

Humberto Debat, Instituto Nacional de Tecnología Agropecuaria (IPAVE-CIAP-INTA), Argentina

Dominique Babini, Consejo Latinoamericano de Ciencias Sociales (CLACSO), Argentina

Latin America has historically led a firm and rising Open Access movement and represents the worldwide region with larger adoption of Open Access practices. Argentina has recently expressed its commitment to join Plan S, an initiative from a European consortium of research funders oriented to mandate Open Access publishing of scientific outputs. Here we suggest that the potential adhesion of Argentina or other Latin American nations to Plan S, even in its recently revised version, ignores the reality and tradition of Latin American Open Access publishing, and has still to demonstrate that it will encourage at a regional and global level the advancement of non-commercial Open Access initiatives.

Access to full-text in English:

https://peerj.com/preprints/27834/

Access to full-text in Spanish:

https://zenodo.org/record/3332621#.XSekx-hKg2z

Cite as: Morrison, H. (2019). Open peer review: A preliminary review, an open offer, observations and discussion. Retrieved from https://sustainingknowledgecommons.org/2019/07/15/open-peer-review-a-preliminary-review-an-open-offer-observations-and-discussion/

Latin America long-time peerless leader in open access

Posted by

2

This post is a preliminary review of Debat & Babini’s preprint PlanS in Latin America: a precautionary note (citation details and links below).

Preliminary review:  Latin America has long been a leader in open access, and had achieved substantially the goals of PlanS more than a decade ago. In 2007, I wrote: “Scielo is an excellent example of what can be accomplished through a nationally subsidized open access program. While the Scielo portal encompasses the scholarly work of many latin countries, Brazil alone, in 2005, brought 160 fully open access journals to the world at a very modest cost of only $1 million dollars” (republished here).

This article is written by experts without peers, and does not really require peer review. This perspective is every bit as worthy of consideration by policy-makers everywhere as anything written by the EU-based OA2020 initiative.

This post will be accompanied by an open offer to conduct a full peer review (with conditions), and some observations on open peer review that are meant to stimulate discussion, however this affirmation of the leadership of Debat, Babini, Scielo & Redalyc warrants a unique post.

Humberto Debat, Instituto Nacional de Tecnología Agropecuaria (IPAVE-CIAP-INTA), Argentina

Dominique Babini, Consejo Latinoamericano de Ciencias Sociales (CLACSO), Argentina

Latin America has historically led a firm and rising Open Access movement and represents the worldwide region with larger adoption of Open Access practices. Argentina has recently expressed its commitment to join Plan S, an initiative from a European consortium of research funders oriented to mandate Open Access publishing of scientific outputs. Here we suggest that the potential adhesion of Argentina or other Latin American nations to Plan S, even in its recently revised version, ignores the reality and tradition of Latin American Open Access publishing, and has still to demonstrate that it will encourage at a regional and global level the advancement of non-commercial Open Access initiatives.

Access to full-text in English:

https://peerj.com/preprints/27834/

Access to full-text in Spanish:

https://zenodo.org/record/3332621#.XSekx-hKg2z

Cite as:

Morrison, H. (2019). Latin America long-time peerless leader in open access. Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://sustainingknowledgecommons.org/2019/07/15/latin-america-long-time-peerless-leader-in-open-access/

Open Peer Review: a Model & an Invitation (2019 update)

Posted by

1

This is a 2019 update of a post originally published in 2005 on The Imaginary Journal of Poetic Economics; the original is republished here. This version reflects experience with open peer review (mine and that of others), further reflection, and research conducted since 2005.

These are some ideas for open peer review that can be used today in experiments that may be helpful to shape future systemic approaches. The overall goal is to facilitate open research by opening up preprints, increase transparency in the peer review process, and to allow peer reviewers to take credit for their work. Interested authors and/or reviewers can experiment with this approach today. For example, an author can post a preprint in a repository, seek volunteer reviewers through a listserv or other social media service for a relevant scholarly community and/or ask a colleague to serve as an editor to coordinate the review process and/or serve as a contact for blind reviews.

Examples and links:

Gibney (2016) wrote an article for Nature on peer review overlay journals built on arXiv that includes links to the journals.

A copy of my peer review of a recent article can be found in the University of Ottawa Institutional Repository here: https://ruor.uottawa.ca/handle/10393/39053 In this way my peer review constitutes a scholarly work with a stable URL that I include in my online CV.

See the Open Access Tracking Project tag oa.open_peer_review for 32 items as of July 15, some posted in the past week.

This model would be compatible with, but does not depend on, peer review overlay journals, featuring an overlay of peer review on articles submitted to and archived in institutional and/or subject repositories, the method recommended in 2009 for the UK for transition to open access in the medium to long term by Houghton et al. (2009) as the most transformative and most cost-effective approach.

The idea of open peer review is not new. While this post will not include a full review of related literature, as one example, Stevan Harnad talks about one approach to open peer review as early as 1996, in Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals.

The goals of this model are:

transparent peer review: unlike blind peer review, readers can see the peer review process in action. Rather than accepting an assessment of certification based on a closed system, readers can judge the peer review process per se, for themselves. This model could accomodate a combination of open and blind peer review – that is, a peer reviewer could publish a signed peer review, or provide comments confidentially, depending on the preferences of authors or the discretion of editors. As an example of the latter, when reviewing opinion pieces in an emotionally heated area, some blind review might be seen as preferable to open peer review.

increased scholarly literacy: it is assumed that a transparent peer review process will facilitate science literacy teaching, as more people will be able to see the peer review process in action

better peer review: exposing the peer review process per se will allow for thoughtful reflection on peer review per se, and facilitate research. This will allow for the development of better and more efficient peer review.

peer-reviewer credit: peer review is an important task, which a great many academics undertake on a voluntary basis. A portfolio of signed peer reviews can be added to the author’s c.v. The best peer-reviewers, those who are thorough, considerate, and respond quickly, can be recognised for their work.

facilitate and recognise author controlled peer review: There are advantages and disadvantages to author-controlled peer review, where the author takes responsibility to seek out peer reviewers. While this is not presently recognised as peer review, it is widely practiced. In the author’s view, an article which has been peer reviewed and edited accordingly prior to submission for publication, is likely to be a better article. Authors who seek out comments from colleagues, and peer reviewers who are sought out by authors, are both demonstrating an openness to collaboration and willingness to listen to critique – both important elements in conducting scholarly research. Author controlled peer review could be used to supplement editor-coordinated peer review (a pre-peer-reviewed article might need only one outside peer reviewer, for example, while an unreviewed work might need two or three).

In some cases, author controlled peer review could be an alternative to editor-coordinated peer review. It would be desirable to develop a set of criteria outlining the optimum for peer review (peer reviewer meets certain criteria, is not a former student, teacher, co-researcher or co-author, at least one peer reviewer from a different cultural background – more important in social than hard sciences – and so forth). Authors should explain whether and how they have met these criteria; this could be accomplished by an automated list, where the relevant criteria are checked off. Some of this could be be automated, as well – for example, a database of the author’s works will reveal former co-authors, and automated comparison of the c.v.’s of author and peer reviewer will reveal common affiliations.

Comments on this blogpost or via e-mail are welcome.

Last updated July 15, 2019.

References

Gibney, E. (2016). Open journals that piggyback on arXiv gather momentum. Nature 530: 7558. Retrieved July 15, 2019 from https://www.nature.com/news/open-journals-that-piggyback-on-arxiv-gather-momentum-1.19102

J. Houghton, B. Rasmussen, P. Sheehan, C. Oppenheim, A. Morris, C. Creaser, H. Greenwood, M. Summers, and A. Gourlay, 2009. “Economics implications of alternative scholarly publishing models: Exploring the costs and benefit” (27 January). Retrieved July 11, 2019 from:

https://pdfs.semanticscholar.org/45dd/cb9ebb9c8505a4ac86718734dda3311f91d8.pdf

See also
Behavioral and Brain Sciences (BBS) Editorial 1978 on Open Peer Commentary Thanks to Stevan Harnad.

Open Peer Review: A Model & An Invitation

Posted by

0

In preparation for some current work in open peer review, this is a re-publication of my August 15, 2005 post on The Imaginary Journal of Poetic Economics (IJPE), drawing from and building on Harnad’s 1996 work in this area. IJPE is live but some readers (including myself while at my office) are reporting that they are not able to connect. Other links have not been tested.

Open Peer Review: A Model & An Invitation

This is one model for an open peer review system. The idea is to automate a great deal of the coordination of peer review, make much of it transparent, and allow peer-reviewers to take credit for their work. This model could fit well with either an institutional repository / peer review overlay approach, or a traditional journal approach for either OA or non-OA journals, or any combination thereof. Readers are welcome to comment, peer-review, and/or experiment with software approaches based on this model [reference to a copyright approach no longer used deleted].

The idea of open peer review is not new. While this post will not include a full review of related literature, as one example, Stevan Harnad talks about one approach to open peer review as early as 1996, in Implementing Peer Review on the Net: Scientific Quality Control in Scholarly Electronic Journals.

The goals of this model are:

transparent peer review: unlike blind peer review, readers can see the peer review process in action. Rather than accepting an assessment of certification based on a closed system, readers can judge the peer review process per se, for themselves. This model could accomodate a combination of open and blind peer review – that is, a peer reviewer could publish a signed peer review, or provide comments confidentially, depending on the preferences of authors or the discretion of editors. As an example of the latter, when reviewing opinion pieces in an emotionally heated area, some blind review might be seen as preferable to open peer review.
increased science literacy: it is assumed that a transparent peer review process will facilitate science literacy teaching, as more people will be able to see the peer review process in action
better peer review: exposing the peer review process per se will allow for thoughtful reflection on peer review per se, and facilitate research. This will allow for the development of better and more efficient peer review.
peer-reviewer credit: peer review is an important task, which a great many academics undertake on a voluntary basis. A portfolio of signed peer reviews can be added to the author’s c.v. The best peer-reviewers, those who are thorough, considerate, and respond quickly, can be recognised for their work.
automate coordination of peer review: it should be possible to establish databases of peer reviewers, most likely distributed databases with central harvesting of key metadata (similar to institutional repositories & OAI), interoperable with other relevant software programs such as publishing software and calendaring systems, to automate much of the coordination of peer review.
peer review improvements through automation: the efficiencies of automation may make it possible to enhance peer review in ways that are not feasible with a system relying largely on one-on-one contact between editor and peer review. For example, there are many good reasons why it might be desirable to seek out an international peer review panel. An automated system would make it possible to easily identify experts in far-away countries, that the editor is unlikely to know personally. It is also possible to think about peer reviewers checking bits of an article, rather than the whole thing. That is, one paragraph of an article may refer to a completely separate area of expertise from the speciality of the author and main peer reviewers; there could be opportunities to ask a specialist to check just the one paragraph, rather than the whole article.
facilitate and recognise author controlled peer review: There are advantages and disadvantages to author-controlled peer review, where the author takes responsibility to seek out peer reviewers. While this is not presently recognised as peer review, it is widely practiced. In the author’s view, an article which has been peer reviewed and edited accordingly prior to submission for publication, is likely to be a better article. Authors who seek out comments from colleagues, and peer reviewers who are sought out by authors, are both demonstrating an openness to collaboration and willingness to listen to critique – both important elements in conducting scholarly research. Author controlled peer review could be used to supplement editor-coordinated peer review (a pre-peer-reviewed article might need only one outside peer reviewer, for example, while an unreviewed work might need two or three).

In some cases, author controlled peer review could be an alternative to editor-coordinated peer review. It would be desirable to develop a set of criteria outlining the optimum for peer review (peer reviewer meets certain criteria, is not a former student, teacher, co-researcher or co-author, at least one peer reviewer from a different cultural background – more important in social than hard sciences – and so forth). Authors should explain whether and how they have met these criteria; this could be accomplished by an automated list, where the relevant criteria are checked off. Some of this could be be automated, as well – for example, a database of the author’s works will reveal former co-authors, and automated comparison of the c.v.’s of author and peer reviewer will reveal common affiliations.

The model

Peer Reviewer Profiles
An academic who is willing to participate in peer review process creates a profile, which could be stored in the institutional repository. Elements of the profile could include:

  • author name
  • affiliation
  • title / position
  • areas of expertise (ideal might be using a standard list)
  • qualifying notes to each area of expertise – e.g., research specialist, practitioner expert
  • links to author’s own works
  • links to samples of work – open, signed peer reviews
  • comments from authors and/or editors
  • comments from recognised experts on the peer-reviewer’s expertise / ability to peer review in a particular area
  • author’s availability – time and number of peer-review requests the author is willing to accept at any given time.
    The time element could potentially be integrated with calendaring systems, e.g. no or fewer requests at particular times
  • author preferences for peer review – e.g. open access and/or fully green journals preferred, professional researchers only, researchers from developing countries welcome, students welcome (in limited numbers, perhaps?)
  • mutuality – in areas of controversy, authors might elect to publish critical reviews from peers with different perspectives, on the condition that their peer mutually publishes the author’s own peer review. This could provide readers with a good service, in alerting them to the existence of alternate viewpoints.

At the Institutional Repository

  • hosting or linking to author profiles and peer review
  • flexibility to accomodate clusters of versions. For example, lead readers first to the final peer-reviewed version, when available, but also make it easy for readers to find the original draft and peer reviewers’ comments.

Publishing software

  • links to author profiles
  • links to peer reviews
  • means of matching available peer reviewers with authors, editors, journals, or other certifying bodies

Comments or peer reviews can be posted as comments or sent to my contact e-mail found here. Any comments or reviews may be incorporated in future versions of this model. Please indicate if you are willing to allow your comments or review to be posted on this blog.

Comments

Peter Suber, August 18, 2005:

Note: Peter wants me to make clear that he does *not* believe that OA depends on peer-review reform, that OA has to wait for peer-review reform, or that OA is valuable primarily for its contribution to peer-review reform. OA is compatible with every kind of peer review and we should pursue it regardless of our position on peer review. (I completely agree, by the way!)

“Just for the record, I believe that peer-review definitely needs improvement and that many promising reforms have exciting synergies with OA. One of my pet ideas (which I wrote about more in the early days than recently) is retroactive peer review. Put the preprint in an OA repository as soon as it’s ready, then apply for review from a journal or free-floating editorial board. If approved, with or without revision, the approved version is also put in the repository with a citation and metadata showing its approved status. So far, this is just an overlay journal. What’s most exciting is the prospect of multiple editorial boards reviewing the same work, say, from different methodological or disciplinary perspectives, with the possibility of each giving (or withholding) its approval, creating something like a market in endorsements and tools that can search and sort by endorsement.”

E-LIS already has many of the components needed
by: Heather Morrison
E-LIS, the open archive for Library and Information Science, already has many of the components that would be needed for an open peer review system. One can already add comments to articles already in the archive – a reviewer could indicate if a comment is intended as a peer review, and link to a Peer Reviewer Profile. All that is needed is some editorial oversight, and communication with the author, and we’re almost there!

An illustration
An illustration of open peer review in action can be found in my Dramatic Growth of Open Access: Revised Update. This illustrates how an update to a peer-reviewed article can be improved, based on helpful constructive criticism on invitation from a friend.

Head and Neck Medicine
Head and Neck Medicine, a new Open Access Journal from BioMedCentral, is planning to follow an open peer review approach. Thanks to Open Access News, Aug. 30.

See also
Behavioral and Brain Sciences (BBS) Editorial 1978 on Open Peer Commentary Thanks to Stevan Harnad.

Last updated September 26, 2005. Minor modifications July 5, 2019.

Cite as:

Morrison, H. (2019). Open Peer Review: A Model & An Invitation. Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://sustainingknowledgecommons.org/2019/07/15/open-peer-review-a-model-an-invitation/

 

Open peer review of “few open access journals are PlanS compliant”

Posted by

0

MDPI’s journal is experimenting with open peer review. Reviewers can choose to make their reviews openly accessible. This recent article that I reviewed has just been published. To read my reviews, click on “review reports” – I am Reviewer 1. If you prefer to skip the details of work that was needed and subsequently done, skip to round 3 for the final review.

Following is the citation and abstract of the article and a portion of my final review that focuses on the work per se, and an update based on subsequent conversation with Frantsvåg. I will publish the copyright statement separately.

Frantsvåg, J.E.; Strømme, T.E. Few Open Access Journals Are Compliant with Plan S.  Publications 2019, 7, 26. https://www.mdpi.com/2304-6775/7/2/26

Abstract

Much of the debate on Plan S seems to concentrate on how to make toll-access journals open access, taking for granted that existing open access journals are Plan S-compliant. We suspected this was not so and set out to explore this using Directory of Open Access Journals (DOAJ) metadata. We conclude that a large majority of open access journals are not Plan S-compliant, and that it is small publishers in the humanities and social sciences (HSS) not charging article processing charges (APC) that will face the largest challenge with becoming compliant. Plan S needs to give special considerations to smaller publishers and/or non-APC based journals.

Excerpt of final round of review:

An open peer review of “Few open access journals are Plan S compliant”: third and final round by Dr. Heather Morrison, Associate Professor, University of Ottawa School of Information Studies, and Principal Investigator, Sustaining the Knowledge Commons, a SSHRC Insight Project. Ó Dr. Heather Morrison, All Rights Reserved (explanation below).

This article presents important research and merits publication; conclusions are basically sound and recommendations appear timely and sensible.

One major substantive point of potential confusion remains. This confusion is evident is PlanS implementation guidance per se which states: “cOAlition S acknowledges that some publishers have established mirror journals with one part being subscription based and the other part being Open Access. Such journals are not compliant with Plan S unless they are a part of a transformative agreement since they de facto lead to charging for both access and publishing in the same way as a hybrid journal does. Funding for publishing in such journals will only be supported under a transformative agreement”. From: https://www.coalition-s.org/implementation/

It is not clear what PlanS is referring to here. The most common arrangement that seems to fit what is described here in my experience is journals that publish both in print (generally on a subscriptions basis) and online (on a fully open access basis, required for inclusion in DOAJ). A journal that is partially open access and partially subscription-based in its online form is a hybrid journal, contrary to PlanS advice.

This confusion is reflected in this article (Table 1 row G and results lines 461-467). Since this reflects the original, this should not be a barrier to publication.

Conflict of interest: although I am an open access advocate and my research focuses on transforming the underlying economics of scholarly publishing in order to sustain open access, I strongly disagree with the PlanS policy approach. In my expert opinion, all open access policy should require exclusively open access archiving. This is the best means to ensure preservation and ongoing open access, particularly in the region for which funders have responsibility. Market-oriented policy is likely to continue or exacerbate a problematic market that for decades has been described as inelastic at best.

Update based on e-mail with Frantsvåg: it appears that what PlanS means by mirror journals is an emerging phenomenon. Elsevier’s Journal of Hydrology X is an example. Following is the explanation of how this works from the Elsevier website. This appears to be an evolution in the hybrid journal model (some content open access, some not). This is an improvement in terms of access as it addresses criticism of the hybrid model on the basis that it is difficult to identify the open access content.

Journal of Hydrology X is the open access mirror journal of Journal of Hydrology

Journal of Hydrology X offers authors with high-quality research who want to publish in a gold open access journal the opportunity to make their work immediately, permanently, and freely accessible.

Journal of Hydrology and Journal of Hydrology X have the same aims and scope. A unified editorial team manages rigorous peer-review for both titles using the same submission system. The author’s choice of journal is blinded to referees, ensuring the editorial process is identical. For more information please refer to our FAQs for authors.

Cite as: Morrison, H. (2019). Open peer review of “Few open access journals are PlanS compliant.” Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://sustainingknowledgecommons.org/2019/04/09/open-peer-review/

 

Open access article processing charges longitudinal study: 2015 preliminary dataset

Posted by

0

We’ve been busy gathering data from publisher websites and other sources since mid-May this year. Our recent check of the top publishers in DOAJ by number of journals illustrates that there is so much change in this area that it’s going to take a while longer to complete this data. In future we’re not even going to attempt a short-term annual census period, rather pursue the gathering of data on an ongoing basis which will permit richer publisher case studies along the way. In the meantime, if anyone can make use of the 2,286 journals that we gathered APC data for or the 103 journals that we gathered article page processing charges for, this plus all the other data we have gathered from various sources, as well as detailed documentation is now available in the OA APC dataverse: http://dataverse.scholarsportal.info/dvn/dv/oaapc/faces/study/StudyPage.xhtml?globalId=hdl:10864/11269&studyListingIndex=2_dc438dc55e8a1e8d291f4fad61a9

We hope this is useful. Informal peer review (please check the documentation for the many limitations to this dataset already noted) in the form of comments to this post are most welcome. We will continue with data gathering and analysis so watch for further posts.

Cite as: Morrison, H. (2015). Open access article processing charges longitudinal study: 2015 preliminary dataset. Sustaining the Knowledge Commons / Soutenir Les Savoirs Communs. Retrieved from https://sustainingknowledgecommons.org/2015/11/12/open-access-article-processing-charges-longitudinal-study-2015-preliminary-dataset/