Administrative procedure for Collaborative and Knowledge-building Project applications
Familiarise yourself with the administrative procedures for calls for proposals for a Collaborative and Knowledge-building Project.
The application review process comprises the two major joint calls for proposals ‘Knowledge-building Project for Industry’ and ‘Collaborative Project to meet Societal and Industry-related Challenges’. We receive around 400 applications for these two calls each year. We also have a number of individual calls that announce funding under the same purpose as this application type.
The calls describe the thematic areas for which research funding can be awarded, as well as the funds available for distribution under each topic. Applicants must tick the thematic areas they wish to be assessed against.
All the applications we receive are directly assigned to the topic the applicant has selected in the application form. Examples of topics are ‘circular economy’ or ‘environmentally friendly energy’. If the applicant has selected several of the topics under the call in their application, we will take this into account in the further processing. Please note that calls for a Collaborative and Knowledge-building Project do not have an ‘open arena’ in addition to the topics given under each call.
Joint overarching requirements and guidelines apply to the calls, and all applicants must comply with these. In addition, each topic has special delimitations and strategic priorities that will be important to the assessment of applications.
The applications are assessed in a four-step process:
- First, the Research Council’s administration carries out a preliminary administrative review to check that the applications meet all formal requirements described in the call.
- The applications are then forwarded to the thematic panels for processing. Each panel generally comprises four or five referees who process five to ten applications in the panel, but the size of the panels and the number of applications they assess will vary somewhat from topic to topic.
- The Research Council’s administration will then consider the extent to which the application is relevant in relation to the call for proposals.
- Finally, the administration carries out a portfolio assessment and presents a recommendation to the relevant portfolio board, which will then make a funding decision.
Click the boxes below for a more detailed description of the different elements of the administrative procedure:
Reception, preliminary administrative review and rejection of applications
As soon as the Research Council has received the applications, it will conduct a preliminary administrative review. Applications that do not meet the formal requirements described in the call may be rejected.
The following formal requirements are checked:
- Is the applicant an approved research organisation?
- Are the application and all attachments written in English?
- Has the project description been attached, and has the correct template been used?
- Has the project manager’s CV been attached?
- Does the project involve at least two Norwegian partners?
- Does the project manager meet the competence requirements?
- Does the application meet the requirements relating to partners’ contribution or cash financing?
- Has a Letter of Intent from the partners been enclosed?
Recruitment of referees
Well before the application deadline, we start the work of finding international referees to consider the applications. However, this work is only complete once the applications have been received and sorted under their respective topics.
The referees we use must meet a number of general requirements:
- Several referees in the panel must be active researchers who have produced a significant body of work in terms of both quantity and quality. In a Collaborative and Knowledge-building Project, panels can include members from outside academia with good industry know-how about the research challenges in question.
- Researchers should have professor qualifications. The minimum requirement is associate professor qualifications or similar.
- The referees’ place of work should preferably be located abroad.
We also aim to avoid using the same panels several years in a row. Ideally, a panel should have some referees who have participated before while others have been replaced.
To find new experts, we use various databases and tools, such as Expert Lookup. We perform automated searches on the basis of project titles and summaries as well as manual searches. We supplement these with searches on other websites such as the Web of Science and Google Scholar, as well as foreign universities with high profiles in the various fields. We also consider the applicants’ own suggestion for suitable referees. The list of sources varies from field to field.
Case officers thoroughly assess each of the potential referees. Our priority is finding good referees, including someone with broad expertise who can be active in discussions and do a good job on giving feedback to applicants. The referees must be able to participate in the panel meeting at the scheduled time and be motivated to do good work, both prior to and during the panel meeting.
We publish the names of all referees after the application assessment process has concluded.
Mapping of referees’ expertise and impartiality
The panels normally comprises four to five referees, and never fewer than three. We endeavour to ensure a good gender balance in the panels. The panel’s overall expertise must cover the breadth of the applications as regards topics, disciplines and industry and sector knowledge, and the axis from basic to applied research. The panels are also put together with a view to the need to consider interdisciplinary applications.
Before the referees are given access to the titles and project summaries of the applications assigned to the panel, they must consent to a confidentiality statement. They then declare their level of expertise on the basis of this information.
The referees indicate one of the following levels of expertise for each application:
- Specialist: The proposal is within your primary area(s) of expertise. You are well qualified to evaluate the proposal.
- Generalist: You have a general knowledge of at least one of the main subjects of the proposal. You are qualified to evaluate the proposal.
- Minor: You have only minor relevant expertise on the main subjects of the proposal.
The Research Council's expertise requirement states that at least two of the referees must have generalist or specialist expertise in the application’s field of research. If the mapping of the panel’s expertise shows that some applications are not adequately covered, we will bring in additional external referees with special expertise. The referees we bring in to assess individual applications will do this on par with the ordinary referee panel members.
In addition to the mapping of expertise, the referees will be sent a list of all project participants mentioned in the applications the panel will assess. This also includes project participants affiliated to partners in the project. To be permitted to assess the application, the referees must declare their impartiality in relation to everyone participating in the project. The referees are asked to pay special attention to the points in the impartiality provisions that concern collaboration, friendships and conflicts.
The referees’ tasks before and during the panel meeting
All panel members read and assess all applications to be considered by the panel. The goal of the panel assessment is to arrive at a consensus-based assessment of the three main criteria Excellence, Impact and Implementation. The assessment consists of numeric marks, in addition to a comment explaining why the panel chose this mark.
An application consists of the application form, project description, the CVs of the project manager and key project participants, as well as Letters of Intent. If the applicant has appended any other information, the Research Council’s administration ensures that these documents are removed from the material sent to the referees to ensure that the applications are assessed on the same grounds.
Before the panel meeting, there is a preliminary meeting, the main purpose of which is to ensure good technical implementation and to give the panel members an opportunity to ask questions. The questions can concern the call itself, the topic or the understanding of or emphasis on the marks.
The panel members submit their individual assessment of each application prior to the panel meeting. The individual assessments will not be forwarded to the applicant, but will be shared between the panel members when everyone has submitted their assessments and form the basis for the discussions during the panel plenum meeting.
For each application, one of the panel members will be assigned special responsibility as principal assessor. The principal assessor usually has specialist expertise in the application’s thematic area, and has a special responsibility for the discussion in the panel meeting and for compiling the panel’s assessments.
The members attend the panel meeting with their different professional views and interpretations of the scale of marks and will also emphasise the various elements of the application differently. The members discuss each application and consider them from various perspectives, and the panel reach a consensus-based assessment and a joint understanding of the scale of marks.
Research Council employees with responsibility for the panels will not participate in the scientific discussion. Their role is to provide guidance and ensure progress, contribute to a joint understanding of the assessment criteria and scale of marks, and ensure that everyone gets to speak. They also handle impartiality according to the rules and stop discussions about matters that fall outside the panel's remit.
Use of threshold values
When the panel have submitted their assessments, the Research Council’s administration calculate the average of the three main marks awarded. The panel’s marks are not weighted.
If the average is lower than 5, the application will be rejected immediately. If the application was awarded ≤3 for one of the main criteria, it will also be rejected. That means that an application that has been awarded the mark 7 for Excellence cannot make up for a mark 2 for Implementation.
The remaining applications will be assessed for relevance.
Assessment of relevance
One important objective for a Collaborative and Knowledge-building Project is to develop R&D competence in Norwegian R&D groups in relation to topics of importance to Norwegian societal actors and Norwegian industry. We also want to stimulate collaboration between the research groups and those representing the societal challenge for which funding is sought. The call and the topics therefore include requirements and guidelines to that end.
The Relevance criterion ensures that the requirements and guidelines described in the call are followed up. The Research Council’s administration carries out the assessment of relevance, which is intended to supplement the referees’ assessments.
In the assessment of relevance, we use a standardised assessment table with the same scale of marks as for the other criteria. A brief written comment will also be included in the feedback to applicants once the application assessment process has concluded.
The assessment table for the Relevance criterion consists of the following points:
- The extent to which the project satisfies the thematic approach the topic requires
- The extent to which the project satisfies the thematic guidelines and delimitations
- The extent to which the project satisfies the requirements and characteristics set out in the call and that otherwise apply to the topic
- The extent to which the project satisfies the requirements relating to project partners
- The extent to which the project satisfies the purpose of knowledge-building in the research groups
if the applicant has selected more than one topic under the call, the application's relevance will be assessed accordingly, and can become relevant for funding in several portfolios.
The overall mark is the average of the marks awarded for the four main criteria, i.e. the referees’ three criteria and the administration’s assessment of relevance. The four main criteria are equally weighted.
Portfolio assessment and decision on allocation
For each topic, our goal is to have a comprehensive project portfolio based on the information given to the applicants through the call and the relevant plans referred to therein. The administration therefore carries out an overall assessment of the portfolio. This assessment does not affect the mark awarded to the applications, but is a tool used to prioritise between applications whose ranking is relatively similar after the assessment of individual applications.
The portfolio assessment takes the following factors into account:
- The applications’ assigned marks based on the assessments
- A good distribution of projects in relation to priorities set out for the specific topic
- The relative volume and quality of grant applications under other calls within the same thematic area in the same application year
- Any changes in the financial or scientific framework set by the ministries
- Priority will be given to projects led by women project managers when the applications are otherwise considered to be on a par
Based on the referee panel’s assessment, the assessment of relevance and the portfolio assessment, the administration writes a recommendation for the portfolio board on whether to grant or reject the application. Applications targeting more than one topic can be included on several lists of recommendations given to different portfolio boards. The Research Council’s administration ensures that the information provided enables the portfolio board to quality assure that the processing of applications has complied with the stipulated procedures, and that all applications have been properly assessed.
The portfolio board is presented with the recommendation documents for all applications, including those defined as being below the threshold value and are therefore in principle not relevant for funding. They are also sent overviews showing the marks awarded to the applications for the individual criteria.
The recommendations (one for each application) are intended to give the portfolio board good insight into the assessments made. The recommendations are also intended to make the portfolio board aware of applications where the assessment of relevance and/or portfolio assessment has had a major bearing on the application’s overall ranking. In this way, the portfolio board can quality assure the administration’s assessments. The portfolio board’s discussion may result in decisions that do not correspond with the recommendation the Research Council has submitted.
The Research Council has 15 portfolio boards with variation in the budgets at their disposal, the number of application types announced, the number of applications for processing, thematic breadth and the expected number of applications granted.
Messages at time of print 19 January 2022, 14:31 CET