MandE NEWS - Jobs Forum

This free service is provided by Rick Davies, editor of the MandE NEWS website.

Please use this Forum to publicise M&E related short term consultancies and full time employed positions.

To publicise training events, use the M&E Training page

To publicise M&E conferences, use the M&E Conferences page

To seek or provide other M&E related information please use the M&E NEWS email list

CAVEAT: The listing of opportunities on this page does not imply any form of endorsement by MandE NEWS of the accuracy of the information provided, or the integrity of the providers.

REQUEST: Please do not waste my time by posting irrelevant material, ether about non-M&E jobs or about other unrelated matters.

MandE NEWS Jobs Forum
Start a New Topic 
Evaluation Consultant Needed - Methodology/Ratings/Indicators/Stats

If interested in this assignment, please send your CV to Barry Kolodkin at!!
Technical Expertise on Evaluation Ratings and Performance Indicators

1. Introduction

The core objective of the Evaluation Department (EvD) of the European Bank of Reconstruction and Development (EBRD) is to contribute to superior institutional performance. It does this through two key channels: by providing a critical instrument of accountability in the form of an objective assessment of performance and results relative to targets; and, by presenting operationally useful findings that contribute to institutional adaptation and learning with respect to future operations. By assessing outputs and outcomes against anticipated results, EvD is able to draw evidence-based conclusions about the Bank’s performance relative to its intended qualitative and quantitative objectives.

EvD, in accordance with Evaluation Cooperation Group practices, utilises ratings in the evaluation of projects. Over time, the methodology, rating scale, nomenclature, and database structure to store the ratings have been modified to reflect greater correlation to OECD evaluation criteria, needs of Board and Management, changes in departmental strategy, and availability of evaluation resources.
Technical expertise is sought to supplement EvD’s analysis of its previous ratings results and data architecture and to prepare itself for ongoing and future assignments.

1. Objective and results

The objectives of the assignment are to:

(i) Further the understanding of the changes in EvD’s ratings methodology and their impact on aggregate performance ratings reported (EvD will provide a comprehensive briefing).
(ii) Bring outside experience, statistical comparison techniques, and knowledge of performance analysis techniques to analyse the existing ratings database and build concordance among different ratings data sets if appropriate
(iii) Prepare an assessment of the advantages and disadvantages of using a 4-point or a 6-point ratings scale, based on experience from outside the EBRD.
(iv) Propose options for developing performance indicators utilising disparate data sets.
(v) Once EvD has selected a “preferred option”, develop a database structure that accommodates the universe of ratings data

2. Scope of the Assignment

The course will likely cover the following topics:

o Evaluation ratings and performance metrics
o Evaluation methodology
o Questionnaires
o Statistics
o Analysing aggregate performance
o Database structures

3. Skills and Experience

The following skills and experience are sought.

o Fluency in evaluation methodology; particularly OECD evaluation criteria and rating methodology
o Strong understanding of evaluation in the context of International Financial Institutions
o Experience utilising ratings to analyse project performance and aggregate performance.
o Skill in statistics, databases and Microsoft Excel

4. Implementation Arrangements, Reporting and Deliverables

4.1 EvD Team

The principal EvD counterpart is Barry Kolodkin, Deputy Chief Evaluator.

4.2 Time phasing/ sequence of events

The assignment will principally be undertaken through 20 working days delivered by the Consultant intermittently over a period of 1-2 months with services provided in the consultants’ home office.

4.3 Deliverables

The consultant will produce the following deliverables. Payment may be invoiced upon the successful completion of each deliverable as deemed by the EvD counterpart. The percentage to be invoiced for each deliverable is listed with its corresponding deliverable.

o An analysis of the existing evaluation database with particular emphasis on differences in ratings from current methodology and previous methodologies. (30%)
o Recommendations on use of rating scales, data fields, performance indicators (30%)
o A database structure in Excel that will join previous and current data sets. (20%)
o A brief wrap-up report. (20%)

Electronic copies of all related material feeding into the Consultant’s report such as data files, statistics, etc. shall be handed over to the EvD key counterpart.

Organisation EBRD

Country United Kingdom