Administrative procedure for the Researcher Project calls in 2021
Learn about the administrative procedure for the Researcher Project calls with application deadlines in February 2021.
First, the applications are assessed by a set of referee panels composed on the basis of the research content of the applications, regardless of which topic(s) the applicant has selected.
The Research Council then considers the application’s relevance to the topic(s) selected by the applicant. The RCN administration presents alternative ranked list to our portfolio boards. The portfolio boards make the final funding decisions.
Click on the boxes below to open them and see a more detailed description of the various elements of the administrative procedure:
Preliminary review and rejection of applications
As soon as the applications have been received, the Research Council conducts a preliminary administrative review. Applications that do not meet the formal requirements described in the call may be rejected.
Assigning the application to a panel
In 2020 and 2021, applications are considered by 29 different panel groups. Each panel group is in turn divided into a certain number of referee panels on the basis of the applications’ scientific scope and the number of applications received under the research area in question.
There will be around 100 panels in total in 2021. The research content of the application will serve as a guideline to which panel it will be assigned to, rather than the topic(s) the applicant has selected. We use a machine learning algorithm to help us with distributing the around 2,500 applications between the different panel groups. The algorithm is trained on the basis of how the applications were assigned in previous years.
It is the case officers of each panel group who decides in which panel the applications will be evaluated. This is to ensure the best possible assessment. In many cases, the applications are discussed before being assigned to a panel group, for example interdisciplinary applications that could be assigned to more than one panel group and applications where it is difficult to find similarities with other applications. In 2020, many of the applications were assigned to a different group than the one suggested by the machine learning algorithm.
Once the applications have been assigned to a group, we organise the applications in panels of 20–30 applications each. The applications in one panel will be within the same scientific fields, which means that 4–7 referees will in total have the expertise to assess most of the applications in the panel. Normally, the same case officer is involved in all parts of the process, from the application is assigned to a panel and until the panel meeting has been held.
Recruitment of referees
The work of finding referees to assess the applications starts already before the call deadline, but cannot be completed until the applications have been assigned to a panel.
The referees we use must meet a number of general requirements:
- Their place of work must be located abroad.
- They must be active researchers who have produced a significant body of work in terms of both quantity and quality.
- They should qualify for professorship. The minimum requirement is qualifications as associate professor or similar.
Expert Lookup is our most important tool for finding referees. We perform automated searches on the basis of project titles and summaries, as well as manual searches. The case officers will then make a thorough assessment of whether we consider the person in question to be a good fit for one of our panels.We supplement the searches in Expert Lookup with searches on other websites such as the Web of Science and Google Scholar, as well as foreign universities with high profiles in the various fields. The list of sources is long and varies from field to field. We also consider the applicant’s own suggestion for suitable referees.
As a rule, we do not use the same referees for more than three years in a row.
We publish the names of all referees and what panel they were on after the entire assessment process has been completed.
The referees who agree to participate in a panel are sent a list of all project managers and project participants in the applications that the panel will assess, and must make a statement on their impartiality towards everyone on the list. We ask them to pay special attention to the points in the impartiality provisions where collaboration, friendships and conflicts are discussed.
Mapping of expertise
The panel must comprise at least four members. The members’ overall expertise must cover the breadth of the applications as regards topics and disciplines, and the axis from basic to applied research. At least one of the panel members must have experience of interdisciplinary research work in the field/fields of research the panel will cover.
The referees declare their level of expertise based on the title, objectives and summary of all applications the panel will assess. Before they receive this information, they must consent to our provisions on confidentiality.
The referees indicate one of the following levels of expertise for each application:
- Specialist (S): The proposal is within your primary area(s) of expertise. You are well qualified to evaluate the proposal.
- Generalist (G): You have a general knowledge of at least one of the main subjects of the proposal. You are qualified to evaluate the proposal.
- Minor (M): You have only minor relevant expertise on the main subjects of the proposal.
The Research Council's expertise requirement states that at least two of the referees must have generalist or specialist expertise in the application’s field of research. If the mapping of the panel’s expertise shows that some applications are not adequately covered, we will find additional external referees with specialist expertise. These referees will assess individual applications and give their assessment on a par with the ordinary referee panel members. In 2020, we had approximately 460 referees divided between a total of 90 referee panels, and an additional 260 referees who helped to assess individual applications.
Information to referees
In connection with the formal appointment of the referee panel, the referees will be informed about the administrative procedure through the document Guidelines for referee panels assessing Researcher Projects 2021 (PDF). The document describes the whole process and the referees’ role before, during and after the panel meeting.
The assessment criteria Excellence, Impact and Implementation and our seven-step scale of marks are explained in more detail We also emphasise the importance of consistent marking and the quality of the feedback to the applicants.
The panel’s tasks before the meeting
All panel members read and assess all applications to be considered by the panel. For each application, two of the panel members will be assigned special responsibility as principal assessor and second assessor, respectively.
The panel members submit their individual assessment of each application prior to the panel meeting. The individual assessments will not be forwarded to the applicant, but will be shared between the panel members when everyone has submitted their assessments and form the basis for their preparations for the discussions during the panel meeting.
- The principal assessor submits a draft for a complete written assessment and a mark for each criterion before the panel meeting.
- The second assessor submits a more brief written assessment and mark for each criterion before the panel meeting..
- The other panel members will submit marks for all criteria before the panel meeting.
Any external referees who are not part of the panel, will submit a complete written assessment and mark for each criterion before the panel meeting.
The panel meeting
The procedures for the panel meeting are described in the Guidelines for referee panels assessing Researcher Projects 2021 (PDF).
The members attend the panel meeting with their different professional views and interpretations of the scale of marks and will also emphasise the various elements of the application differently. The members discuss the application and consider it from various perspectives. The panel reach a consensus-based assessment and a calibrated scale of marks, and prepare a ranked list of the best applications.
Since the applications are assessed by a set of panels appointed on the basis of the research content, the quality of the project will be assessed without regard to the thematic priorities defined under each topic in the call for proposals.
The person responsible for the panel representing the Research Council will not participate in the scientific discussion. Their role is to provide guidance and ensure progress in the meeting, contribute to a joint understanding of the assessment criteria and scale of marks, and that everyone gets to speak. They also handle impartiality according to the rules, stop discussions about matters that fall outside the panel's mandate and proofread the final written assessments.
Assessment of relevance
The Research Council assesses the application’s relevance to the selected topic. Applications that target the topic Ground-breaking Research (FRIPRO) are not assessed for relevance.
Whereas the panel’s assessment of the application’s impact addresses the project’s general potential for scientific and societal impact, the application will also be assessed for relevance to the priorities described under each topic in the call for proposals. These priorities are based on the ministries’ allocation letters to the Research Council.
The panel’s overall assessment of the application must exceed the threshold value stipulated for each topic in order to be considered for relevance. If an applicant has selected several topics for their application, independent assessments of relevance will be carried out for each of the topics.
We use the attachment Relevance to the topic, the project description and the referee panel's assessment of the application as the basis for our assessment.
When the portfolio assessment begins, the applications have been awarded five separate marks and written assessments for excellence, impact, implementation, overall assessment and relevance, respectively (applications that target the topic Ground-breaking Research (FRIPRO) are not assessed for relevance and are given four marks). In addition, the best applications have been ranked by the panel.
When we prioritise between applications when preparing our ranked lists for the portfolio boards, we carry out a portfolio assessment that takes account of the following:
- The applications’ assigned marks based on the assessments. The panel's overall mark is given more weight than the mark awarded for relevance..
- A good distribution of projects in relation to priorities set out for the specific topic.
- The relative volume and quality of grant applications within the same topic under other calls in 2021.
- Any changes in the financial or scientific framework set by the ministries.
- Priority will be given to projects led by women project managers when the applications are otherwise considered to be on a par.
For applications targeting Ground-breaking Research (FRIPRO), the panel’s ranking is also important in the portfolio assessment.
All these factors are included in the portfolio assessment, which forms the basis for the ranking lists we present to the portfolio boards.
Decision on allocation
The decision on which applications to grant are made by the portfolio boards. Only applications that are awarded a mark for overall assessment of 5, 6 or 7 by the panel will be considered for funding.
We have 15 portfolio boards with great variation in the budgets at their disposal, the number of application types announced, the number of Researcher Project applications for processing, thematic breadth and the number of applications granted. The different boards will place different emphasis on the factors mentioned above (see the section ‘Portfolio assessment’).
Ground-breaking Research (FRIPRO) applications are granted by the three thematic portfolio boards. FRIPRO’s objective is to fund the very best applications, with particular emphasis on scientific quality, regardless of the topic. This means that the mark awarded for Excellence is given particular weight and that the portfolio board will also take special account of the application’s ranking by the panel when deciding whether or not to grant funding.
The other portfolio boards with responsibility for the defined thematic portfolios can give different weight to the different factors, but never at the expense of the priorities described under the relevant topic in the call for proposals.
The administration often submits alternative ranking lists to the portfolio board where these factors are weighted differently. The lists form the basis for the portfolio board’s discussion of what considerations they want to prioritise, within the budget available to them and the priorities described in the call. The portfolio board’s discussion may result in decisions that do not correspond with the proposals the Research Council has submitted.
Applications targeting more than one topic can be included on several lists of recommendations given to different portfolio boards. If more than one portfolio board grants the application, it will nevertheless only be funded by one of the portfolio boards. Another application will then be moved up from the waiting list and granted the funding that has now become available.
Feedback to applicants
In 2021, all applicants will receive feedback in the form of marks and written assessments from the joint panel.
In 2020, the applications that did not move past the qualification stage and were therefore not assessed by the joint panel, which accounted for around 30% of applications, only received the principal assessor’s written feedback as scientific feedback. However, the marks awarded by all the panel members nevertheless determined whether an application passed the qualification stage. This procedure was implemented as a special measure because of the coronavirus situation and will not be continued in 2021.
In this year’s call, we will announce which applications have been granted funding on our website and in our newsletter in week 25.
Applicants whose application has been rejected will receive a rejection letter via My RCN Web as soon as possible.
As a result of the extra funding awarded for excellent research and challenge-driven humanities research, we will also grant a few applications in July/August 2021.
Funding from the Fellesløft IV joint funding initiative will be awarded according to a different process where the final decisions are made in the autumn. This process is described in the call for proposals.