2021 Marie Curie Postdoctoral Fellowship (HE-MSCA-PF-2021)

[aspil]

Thanks for the add-on!
I think strategically this is indeed crucial for msca evaluation: “respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire”

This is quite important. For the Horizon 2020 programme, there was an NCP Handbook which lists the important points to be addressed under each subheading. I followed that for my first submission and came very close to a SoE.
I could not find something like for the revised format under Horizon Europe.

[anonymous_insider]

Dig out a post from 2019 (page 63), might be inspiring somehow

This is the most useful thing to understand. Way more important than the famous magic link (in my personal opinion and experience). It also makes the point about possible delays in sending the ESR.
Namely, since there must be 3 evaluators submitting an IER (1) it's not easy to find the required number of experts, (2) some experts may become overloaded because of their broad competences, (3) at some point some expert might withdraw (lack of time, conflict of interest) or might be revoked for an emerging conflict of interest. This to say that the remote evaluation phase (step 1 to produce 3 IERs/proprosal) will take time and it's the phase that causes the major bottleneck.
Once all the IERs are in place, the consensus panel establishes the final scores and from there they rank the proposals from the highest to the lowest score. Meaning that all the proposals are ranked, hence if our proposal is now in 'ranking' mode in the magic link, it means that the evaluation has been completed and the EC is ranking the proposals. The cut-off score for funding depends on the available budget.
Another fundamental point is respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire. Remote evaluators are not given much time (in general, less than 1 working day per proposal); and as meticulous as they may be, some information might remain unnoticed or be perceived as unconvincing. The worst case is when information is misplaced. Remote evaluators need to answer specific points in a questionnaire and if the info related to one sub-criterion e.g. of excellence was written in impact, they must evaluate the sub-criterion according to the information at hand.
I hope this helps

Thanks for the add-on!
I think strategically this is indeed crucial for msca evaluation: “respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire”

In fact. I have learned this thanks to the rejection of my very first MSCA application. Eventually, I have realized how crucial this is when I served as remote evaluator. When information was misplaced, I felt so bad I had to score the proposal lower. On a side note, for the sake of objectiveness I have considered that information even if misplaced. But I assume it's only because I empathize with the applicants following my rejections Don't take this for granted, some reviewers are tough (to use a nice word...).

[anonymous_insider]

Thanks for the add-on!
I think strategically this is indeed crucial for msca evaluation: “respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire”

This is quite important. For the Horizon 2020 programme, there was an NCP Handbook which lists the important points to be addressed under each subheading. I followed that for my first submission and came very close to a SoE.
I could not find something like for the revised format under Horizon Europe.

The template headings also point to specific keywords in the sub-criteria questionnaire. But I think that you may also find the remote evaluator questionnaire online, it should be openly available for a matter of transparency of the evaluation process.
On a side note: the new template killed me... Some sub-headings were taking 4+ lines and you can't get rid of the text in parentheses

[Abz]

Post by ENV2021 » Sun Jan 26, 2020 4:30 pm

Hi there,
a friend of mine is an MC evaluator and I would like to share with you what he explained to me about the evaluation process.
Three evaluators per proposal plus a chair that supervises the process. Each evaluator may evaluate a few proposals (10-15).
Each evaluator produces a report individually without sharing his comments; only when the three evaluators have submitted their reports it starts a consensus phase when the evaluators discuss their comments about the proposal. When they reach a consensus, and only then, they propose the score, always under the supervision of the chair. They are not allowed to see the evaluation report of the previous submitted proposal (if any) nor to know the old score, actually, this report remains undisclosed; the chair may ask the evaluators to rediscuss specific points if he considers it necessary.
Evaluators have to follow a specific point list and that is why it is so important to fill out the proposal including each and every point specified in the guideline, being specific, concrete. And you have to say the right things where they are expected to be in the proposal, not elsewhere in the text. If you state something in the "excellence" part that should have been stated in the "impact" part, well...that is your fault.
Regarding the CV, effectively the number of papers doesn't care, as it said in the guideline you may be penalised only if according to your research career you have not produced enough; evaluators check your CV to have an idea of what you have done before, especially to determine if your training programme is appropriate and if you effectively are going to learn something new which is the most important thing in this fellowship, besides project excellence. He also told me that many of the projects he reviewed were excellent but even in this case if you fail addressing all and each point they have to start to decrease the score, and few tenths of a point means getting or not the fellowship or even getting a bad score.
Said that..of course, subjectivity is impossible to remove but at least I think this is the most well-designed evaluation process I know.

Dig out a post from 2019 (page 63), might be inspiring somehow

This is the most useful thing to understand. Way more important than the famous magic link (in my personal opinion and experience). It also makes the point about possible delays in sending the ESR.
Namely, since there must be 3 evaluators submitting an IER (1) it's not easy to find the required number of experts, (2) some experts may become overloaded because of their broad competences, (3) at some point some expert might withdraw (lack of time, conflict of interest) or might be revoked for an emerging conflict of interest. This to say that the remote evaluation phase (step 1 to produce 3 IERs/proprosal) will take time and it's the phase that causes the major bottleneck.
Once all the IERs are in place, the consensus panel establishes the final scores and from there they rank the proposals from the highest to the lowest score. Meaning that all the proposals are ranked, hence if our proposal is now in 'ranking' mode in the magic link, it means that the evaluation has been completed and the EC is ranking the proposals. The cut-off score for funding depends on the available budget.
Another fundamental point is respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire. Remote evaluators are not given much time (in general, less than 1 working day per proposal); and as meticulous as they may be, some information might remain unnoticed or be perceived as unconvincing. The worst case is when information is misplaced. Remote evaluators need to answer specific points in a questionnaire and if the info related to one sub-criterion e.g. of excellence was written in impact, they must evaluate the sub-criterion according to the information at hand.
I hope this helps

I don't think this helps because it is stating the obvious and we all know about how the evaluation is done. The question we are engaging with when thinking about the timing of the ranking is: are phase changes done according to a pattern? Other questions we think about include: Are the scores of successful applicants entered first? What do the time gaps between phase changes (from Submission to Evaluation to Ranking) mean? Because we don't have full access to the information, we are trying to figure out from people's past experiences. The evaluations are done during the first six weeks of the evaluation process and what we are thinking about is what happens during the last week.

[Aio]

Still in Evaluation
Anyone in my situation?
I have asked for an extra stay in the insdustrial sector...I wonder if it was a mistake, as more funding is required. I cannot help thinking about all mistake I might have done...I am very sad as I was hoping to pass at least 70 point (this is my last chance because of the restriction timing after PhD). I was sure I was not getting it but I feel like shit that I did not reach the threshold.

Last year I was also in evaluation, 66 points.

Me too!

[anonymous_insider]

Dig out a post from 2019 (page 63), might be inspiring somehow

This is the most useful thing to understand. Way more important than the famous magic link (in my personal opinion and experience). It also makes the point about possible delays in sending the ESR.
Namely, since there must be 3 evaluators submitting an IER (1) it's not easy to find the required number of experts, (2) some experts may become overloaded because of their broad competences, (3) at some point some expert might withdraw (lack of time, conflict of interest) or might be revoked for an emerging conflict of interest. This to say that the remote evaluation phase (step 1 to produce 3 IERs/proprosal) will take time and it's the phase that causes the major bottleneck.
Once all the IERs are in place, the consensus panel establishes the final scores and from there they rank the proposals from the highest to the lowest score. Meaning that all the proposals are ranked, hence if our proposal is now in 'ranking' mode in the magic link, it means that the evaluation has been completed and the EC is ranking the proposals. The cut-off score for funding depends on the available budget.
Another fundamental point is respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire. Remote evaluators are not given much time (in general, less than 1 working day per proposal); and as meticulous as they may be, some information might remain unnoticed or be perceived as unconvincing. The worst case is when information is misplaced. Remote evaluators need to answer specific points in a questionnaire and if the info related to one sub-criterion e.g. of excellence was written in impact, they must evaluate the sub-criterion according to the information at hand.
I hope this helps

I don't think this helps because it is stating the obvious and we all know about how the evaluation is done. The question we are engaging with when thinking about the timing of the ranking is: are phase changes done according to a pattern? Other questions we think about include: Are the scores of successful applicants entered first? What do the time gaps between phase changes (from Submission to Evaluation to Ranking) mean? Because we don't have full access to the information, we are trying to figure out from people's past experiences. The evaluations are done during the first six weeks of the evaluation process and what we are thinking about is what happens during the last week.

I have spent and keep spending sleepless nights thinking of the same questions... I can share with you that in general rejections are sent out first (at least, this has been my personal experience so far).
Regarding 'ranking' vs 'evaluation', I have the feeling that this simply reflects the progress bar in the timeline graph. However, if some people are still in 'evaluation' while others are in 'ranking', there might be a reason.

[dw2022]

Thanks for the add-on!
I think strategically this is indeed crucial for msca evaluation: “respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire”

This is quite important. For the Horizon 2020 programme, there was an NCP Handbook which lists the important points to be addressed under each subheading. I followed that for my first submission and came very close to a SoE.
I could not find something like for the revised format under Horizon Europe.

The template headings also point to specific keywords in the sub-criteria questionnaire. But I think that you may also find the remote evaluator questionnaire online, it should be openly available for a matter of transparency of the evaluation process.
On a side note: the new template killed me... Some sub-headings were taking 4+ lines and you can't get rid of the text in parentheses

I can echo on the last note and I think somewhere in the guidance said the subtitle should remain intact in the submitted proposal…

[Serpentina]

Maybe it is too early yet, but have anyone asked the Spanish NCP today about the results?

[anonymous_insider]

Dig out a post from 2019 (page 63), might be inspiring somehow

This is the most useful thing to understand. Way more important than the famous magic link (in my personal opinion and experience). It also makes the point about possible delays in sending the ESR.
Namely, since there must be 3 evaluators submitting an IER (1) it's not easy to find the required number of experts, (2) some experts may become overloaded because of their broad competences, (3) at some point some expert might withdraw (lack of time, conflict of interest) or might be revoked for an emerging conflict of interest. This to say that the remote evaluation phase (step 1 to produce 3 IERs/proprosal) will take time and it's the phase that causes the major bottleneck.
Once all the IERs are in place, the consensus panel establishes the final scores and from there they rank the proposals from the highest to the lowest score. Meaning that all the proposals are ranked, hence if our proposal is now in 'ranking' mode in the magic link, it means that the evaluation has been completed and the EC is ranking the proposals. The cut-off score for funding depends on the available budget.
Another fundamental point is respecting the template and placing the relevant information exactly where it is expected to be, possibly using bold for specific keywords that are part of the evaluation questionnaire. Remote evaluators are not given much time (in general, less than 1 working day per proposal); and as meticulous as they may be, some information might remain unnoticed or be perceived as unconvincing. The worst case is when information is misplaced. Remote evaluators need to answer specific points in a questionnaire and if the info related to one sub-criterion e.g. of excellence was written in impact, they must evaluate the sub-criterion according to the information at hand.
I hope this helps

I don't think this helps because it is stating the obvious and we all know about how the evaluation is done. The question we are engaging with when thinking about the timing of the ranking is: are phase changes done according to a pattern? Other questions we think about include: Are the scores of successful applicants entered first? What do the time gaps between phase changes (from Submission to Evaluation to Ranking) mean? Because we don't have full access to the information, we are trying to figure out from people's past experiences. The evaluations are done during the first six weeks of the evaluation process and what we are thinking about is what happens during the last week.

Maybe 'ranking' means that the ESR has been uploaded but not yet published (i.e., it's still in the back-end). Likewise, 'evaluation' might mean that the ESR has not been uploaded yet.

[dw2022]

Since Dreamer said his NPC is in Spain, maybe users with NPC in Spain could ask them tomorrow after system maintenance is over (around mid-day)?

Shout-out to people applying to go to Spain. Help fellow applicants

I'm Spanish applying for Belgium. I'll call tomorrow just in case we can get some info

digging this post out