# Evaluation manager process

{% hint style="info" %}
***Update Feb. 2024:*** We are moving the discussion of the *details* of this process to an internal Coda link ([here](https://coda.io/d/The-Unjournal-Hub-internal_d0KBG3dSZCs/Evaluation-Management_suhOX#_luMy4), accessible by team members only). We will present an overview in broad strokes below.
{% endhint %}

{% hint style="info" %}
See also [mapping-evaluation-workflow](https://open-2c.gitbook.com/url/globalimpact.gitbook.io/the-unjournal-project-and-communication-space/policies-projects-evaluation-workflow/mapping-evaluation-workflow "mention")for an overview and flowchart of our *full* process (including the evaluation manager role).
{% endhint %}

***Compensation:** Evaluation managers are compensated a minimum of $300 per project, and up to $500 for detailed work. Further work on curating the evaluation, engaging with authors and evaluators, and writing detailed evaluation summary content can earn up to an additional $200.*

{% hint style="info" %}
**If you are the evaluation manager please follow the process described in our private Coda space** [**here**](https://coda.io/d/The-Unjournal-Hub-internal_d0KBG3dSZCs/Eval-Mgmt-7-step-process-guide_suNn2#_lu-8l)
{% endhint %}

### **In brief, evaluation managers:**

1. Engage with our previous discussion of the papers; why we prioritized this work, what sort of evaluators would be appropriate, what to ask them to do.
2. Inform and engage with the paper's *authors*, asking them for updates and requests for feedback*.* The process varies depending on whether the work is part of our [direct-evaluation-track](https://open-2c.gitbook.com/url/globalimpact.gitbook.io/the-unjournal-project-and-communication-space/policies-projects-evaluation-workflow/considering-projects/direct-evaluation-track "mention") or whether we require authors' permission.
3. Find potential evaluators with relevant expertise, contact them. We generally seek two evaluators per paper.
4. Suggest research-specific issues for evaluators to consider. Guide evaluators on our process.
5. Read the evaluations as they come in, suggest additions or clarifications if necessary.
6. Rate the evaluations for awards and bonus incentives.
7. Share the evaluations with the authors, requesting their response.
8. Optionally, provide a brief "evaluation manager's report" (synthesis, discussion, implications, process) to accompany the evaluation package.

See also:

{% content-ref url="management-process/choosing-evaluators" %}
[choosing-evaluators](https://open-2c.gitbook.com/url/globalimpact.gitbook.io/the-unjournal-project-and-communication-space/management-tech-details-discussion/management-process/choosing-evaluators)
{% endcontent-ref %}

See also: [protecting-anonymity](https://open-2c.gitbook.com/url/globalimpact.gitbook.io/the-unjournal-project-and-communication-space/policies-projects-evaluation-workflow/evaluation/protecting-anonymity "mention")

### Some other important details

1. We give the authors two weeks to respond before publishing the evaluation package (and they can always respond afterwards).
2. Once the evaluations are up on PubPub, reach out the evaluators again with the link, in case they want to view their evaluation and the others. *The evaluators may be allowed to revise their evaluation, e.g., if the authors find an oversight in the* evaluation. (We are working on a policy for this.)
3. At the moment (Nov. 2023) we don't have any explicit 'revise and resubmit' procedure, as part of the process. Authors are encouraged to share changes they plan to make, and a (perma)-link to where their revisions can be found. They are also welcome to independently (re)-submit an updated version of their work for a later *Unjournal* evaluation.