Victor Merriman writes:
The short answer is that the international academic standard for excellence – rigorous peer review – should be applied, and subject associations should be approached to test levels of interest in piloting PR peer review networks. This would have real advantages for the reliability of research assessment exercises, and benefits should accrue to those engaged in processes, outcomes, and – as appropriate – research outputs, also.
Drama and Dance (REF UoA 35) does not overtly rank journals or academic publishing houses for purposes of grading degrees of excellence, as is the case in some other disciplines. There is a peer group consensus that this is a good policy, enabling a broad range of work, and working circumstances, to be put forward for evaluation. It does also burden evaluation panels by requiring that they perform rigorous assessments of quality more or less ab initio, and inconsistencies in presenting PR outputs can, understandably generate systemic problems of evaluation, not least of consistency of judgement, and assessment narratives.
The problems appear to deepen where research underpinning Impact claims is under consideration, and chairs of sub-panels (Delgado) and main panel (Brown) have gone on record since the REF 2014 results were published, expressing frustration at the blurring of lines between Practice and Practice Research in outputs classified below the threshold required to support Impact claims.
Across performance sub-disciplines, the matter has attracted attention to the extent that in recent issues of Studies in Theatre and Performance (Routledge), a proposal to establish a section of that journal, ‘Curating Practice-as-Research’ (Hann and Ladron de Guevara, 35.1: 3-6, 2015(a)) led to ‘A General Call for PaR Contributions’(Hann and Ladron de Guevara, 35.2: 159-160, 2015 (b)).
HOW SHOULD WE PEER-REVIEW PRACTICE RESEARCH?
This brings into focus the matter of how peer review might play out in practice, and how its processes might be, themselves, demonstrably rigorous and reliable.
Each discipline will have its own wisdom, practices and conventions which will be reflected in the ways in which this might be addressed. In the interests of furthering the discussion, I will outline an example of the nature and role of peer review in relation to a piece of Performance Practice Research, as follows:
Establishment of Peer Review Network: Subject associations – ideally in concert with HEFCE – would solicit interest in joining a peer-review network from Practice Researchers in a discipline. They would be selected on the basis of CV and statement of areas of expertise.
Identification of peer reviewers: A Researcher proposes a project, to be carried out over a specified period of time, notifies the network, which posts a call for peer reviewers, as appropriate (one PR researcher; one practising artist, for example).
Clarification of research context, and imperatives: Peer reviewers comment in blind review
Clarification of processes (to include research methods, documentation, and dissemination strategies): Peer reviewers comment in blind review
Engagement with process: Negotiable. Note that, depending on the nature of the work, this might make blind review impossible, and would require a more open, dialogical, process. It is not difficult to establish the rigour of such processes, as is the case for QAA panels conducting institutional audit, for example, and clear statements as to ensuring rigorous review would need to be made available.
Review of outcomes: Optional.
Review of outputs: Essential.
At the conclusion of the process, research processes and/or outputs which satisfy the conditions negotiated during peer review would be entered in an index of work done over a given period. This could be done by archiving what Hann and Ladron de Guevara (2015 (b): 159) refer to as a ‘curated portfolio’.
Once indexed, the quality of the work would be established, and, in the case of Impact claims, submitting institutions and evaluation panels could concentrate on establishing the case for impact, rather than revisiting possible problems around the status of the research.
One of the issues identified by Professor Bruce Brown, Chair, Main Panel D (REF2014) was the relatively low number of PR outputs submitted to the recent exercise (37%). He suggested that this might be because of a lack of confidence on the part of researchers, and, even more so, on the part of institutions, increasingly swayed by risk-averse research managers seeking to maximise institutional funding and reputation.
Perhaps the suggestions in this note will contribute to a debate by means of which the aspirations and practices of practice researchers and institutional interests come closer to each other.
Victor Merriman, Professor of Performing Arts, Edge Hill University
Use of photograph courtesy of Edge Hill.
Rachel Hann and Victor Ladron de Guevara (2015(a)), ‘Addressing Practice: introducing a new section for STP (Studies in Theatre and Performance, 35.1: 3-6)
Rachel Hann and Victor Ladron de Guevara (2015(b)), ‘Curating Practice-as-Research – A General Call for PaR Contributions’(Studies in Theatre and Performance, 35.2: 159-160).