British Academy: The UK's National Academy for the Humanities and Social Sciences
Monitoring and evaluation policy
(i) Policy on Monitoring and Evaluation
The principles underpinning the Academy's policy for monitoring the outcome of awards and evaluating the success of its programmes are that procedures should be cost-effective, mainly qualitative, and deliberately proportionate to the scale of funding on offer.
End of grant reports: All recipients of research awards are required to submit an end of grant report. The exact requirements in terms of length and detail required may vary from scheme to scheme, and are set out in the conditions of award for each scheme. The minimum requirements for a report on the use of a grant are:
- account of research carried out
- identification of advances in knowledge or understanding resulting from the research
- details of publications/other dissemination
- financial statement of costs incurred with evidence of expenditure (not relevant for salary-only awards paid by agreed schedule)
- confirmation of deposit of digital resources, or waiver (where relevant)
Assessment of reports: Reports on research are sent out to assessors within the Fellowship for evaluation. Holders of grants of more than one year's duration are required to submit annual progress reports for approval before subsequent instalments of grant are released. Reports on conference grants are scrutinised within the office and any queries forwarded to assessors. Reports on Academy Research Projects are assessed by the Projects Committee, and each Academy Research Project is, in addition, rigorously reviewed every five years. Reports on research funded through the International Policy Committee are scrutinised by the office and any queries forwarded to assessors.
Reports are graded on the scale satisfactory/unsatisfactory/further information required. A research report must have been classified as satisfactory before any further applications for funding from the principal investigator are accepted. The Academy deliberately eschews any finer-grained assessment scale for reports in order to maintain the principle of proportionality.
Compliance: Reminders are issued to those who have not yet reported three months after the completion date of a grant, and the opportunity granted for award-holders to meet the conditions of award, or to bring exceptional extenuating circumstances to the Academy's attention. Persistent failure to comply with the conditions of award will result in the Academy writing to the Vice-Chancellor of the scholar's institution, and lead to the possibility of sanctions.
The sanction for non-compliance with the conditions of award is penalty-listing: that is, to disbar, sine die, the scholar concerned from applying to the Academy under any of it grant-giving schemes; and, if the grant is not refunded, to disbar, for a period of two years, all members of that institution from applying to the Academy under any of its grant-giving schemes. In cases of grants awarded to independent scholars or those who have left academic life without complying with the conditions of award, the sanction is to disbar, sine die, the individual from applying to the Academy under any of its grant-giving schemes. The Academy may seek to reclaim any part of a grant that is not adequately accounted for in scholarly terms. Each awarding Committee is presented with the details of those scholars who have failed to report, and decides whether to proceed to the penalty-listing stage. The Research Committee receives a periodic statement listing non-compliant individuals who have received funding via any Academy Committee including the Board for Academy-Sponsored Institutes and Societies.
Detailed procedures governing the steps to be followed before sanctions are implemented are set out in section ii. Removal from the penalty-list can be achieved at any time through compliance with the conditions of award, or refund of the grant; and there is a simple procedure for appeals by an individual or an institution to be considered.
Publications: Award-holders are required to provide the Academy with bibliographical information of all published outputs resulting from funded research including books, articles and any non-print outputs. One copy of any book resulting from funded research should be presented to the Academy for lodging in its Library, which thereby serves as an archive of research supported by the Academy. Significant digital resources created as a result of research must be offered for deposit with the Arts and Humanities Data Service or Economic and Social Data Service, and compliance will be monitored. Failure to comply, subject to any waiver agreed with the relevant Data Service, will result in penalty-listing (see above).
Impact measures: The Academy considers that the value of the research it funds is assessed through rigorous peer review at application stage. Owing to the very large volume of grants awarded (nearly 2,000 awards each year), the relatively small financial cost of the awards, and the pressure on the Academy's publicly-funded administrative and other resources, there are currently no plans to introduce a formal system of impact measurement such as a citations index.
Evaluation of Programmes: The Academy monitors the outcomes of each of its schemes in terms of distribution of applications and awards on a range of criteria including age, gender, institutional affiliation, subject classification, grade awarded, and national base. Additionally, each programme is subject to periodic, usually annual, review by the Committee responsible for its administration. The published guidance for applicants and conditions of award for each scheme are reviewed, and may be amended, annually. Once revised, these regulations remain operative for the whole of the subsequent academic session. From time to time, a survey of a cohort of applicants may be undertaken (a) to ascertain further information about the progress of research/development of careers initially funded by the Academy; (b) to ensure that the Academy's schemes are meeting the needs of the academic community in terms of service delivery; and (c) to ensure that the programmes offered by the Academy are judged to add value within the spectrum of public funding available nationally.
(ii) Procedures for Penalty-listing
The Academy has adopted the principle of penalty-listing a person and/or an institution for failing to comply with the conditions of award.
The procedures to be followed before penalty-listing are set out below,
End of grant report due
(a) Report submitted --> Sent for assessment --> If deemed satisfactory --> File closed
If deemed unsatisfactory --> Further information sought
If further information satisfactory --> Close file
If further information unsatisfactory --> Pre-sanction
(b) Report not submitted --> Reminder(s) sent --> If report then submitted --> Follow (a) above
If no report submitted --> Letter from Secretary to Vice-Chancellor (where relevant)
If report then submitted --> Follow (a) above
If no report submitted in response to Letter to Vice-Chancellor (or award-holder independent) --> Pre-sanction
Pre-sanction
(c) Matter placed before awarding Committee to decide whether sanction appropriate
Sanction options
(d) If award-holder at institution --> Request refund of grant
If refunded --> Close file.
No further sanction
If no refund received --> penalty-listed: all members of staff at that institution disbarred from applying to the Academy for two years. Individual disbarred from applying to the Academy sine die
(e) If award-holder independent --> Request refund of grant
If refunded --> Close file.
No further sanction
If no refund received --> penalty-listed: individual disbarred from applying to the Academy sine die
Failure to deposit digital resources with a Data Service or to obtain a waiver (where relevant) also constitutes non-compliance with the conditions of award. Such cases will automatically go to the 'pre-sanction' stage of the process outlined above.
Cross-Committee sanction
The sanction of disbarring an individual or institution from applying for Academy support applies across the board, that is, to all grant-funding streams, including those that flow through the Board for Academy-Sponsored Institutes and Societies (BASIS) and the International Policy Committee. A central list is kept on a file that is accessible to relevant staff. Assistant Secretaries, including staff at BASIS institutions, are responsible for ensuring information is shared and made available to awarding Committees as appropriate.
Removal from Penalty-list
Removal from the penalty-list can be achieved at any time through submission of a satisfactory report, or the refund of the grant awarded. If an individual or institution wishes to appeal against the sanction, they may submit a written case, for evaluation by the relevant awarding Committee.