Programme Partnership Arrangement Evaluation 2014: A primary research feedback report | Oxfam | Evaluation

Please download to get full document.

View again

of 21
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report



Views: 5 | Pages: 21

Extension: PDF | Download: 0

Related documents
font-size: 10pt
  MARCH 2015 Programme Partnership  Arrangement Evaluation 2014  A primary research feedback report Coffey International  PPA Evaluation 2014 2 Oxfam summary feedback report 1 Introduction The following feedback report covers the primary research with Oxfam GB as part of the 2014 PPA evaluation. The report provides a short summary of findings from this research period. Please note that this feedback report is not an evaluation report and is not designed to provide in-depth analysis or an assessment of Oxfam GB’s performance. Instead, findings from the report will be used in conjunction with other data sources as part of the Fund level evaluation. No uses of quotes in the Fund level report will be used without Oxfam GB’s prior approval, nor will the report be included as a case study or annex in the 2014 PPA Evaluation Report. We have invited Oxfam to respond to points of accuracy and interpretation from this feedback report as part of good validation practice and our commitment to shared learning and collaborative working. 1.1 Background to the 2014 PPA Evaluation The 2014 PPA evaluation examines all 41 PPA holders and £360 million of funding over the three years of this round of PPA funding (2011 through April 2014). The 2014 PPA evaluation is a fund-level evaluation that seeks to identify and explain the effects of the PPA portfolio and the added value of strategic funding. This means that the evaluation is focused on identifying and analysing patterns of effects and changes across the fund. This also means that agencies are not being subject to a performance assessment or individual assessment. The focus of the 2014 PPA evaluation is to better understand the effects and impacts achieved from the current round of PPA funding, particularly understanding how the flexible nature of PPA funding has influenced the type of investment decisions that organisations take and the extent to which these investment decisions are of strategic importance for the organisation, its countries of operation, the sector(s) in which it works, and ultimately how these investment decisions achieve results for intended beneficiaries.  A revised Theory of Change to frame the PPA evaluation was developed in close cooperation with PPA agencies throughout the autumn of 2013 which then informed a revised Evaluation Strategy and Evaluation Framework that was circulated in May 2014. This Evaluation Strategy builds on our lessons learnt from both the 2012 PPA Mid Term Evaluation and the 2013 GPAF Mid Term Evaluation, particularly the additional learning that comes from engaging directly with agencies in an evaluation process that is clear, highly consultative and proportionate. 1.2 Rationale for conducting primary research with a sample of PPA agencies Our rationale for conducting primary research with a sample of agencies was driven both by the structure of the Evaluation Framework and by our experience of how to work more effectively with agencies. The 2014 Evaluation Framework is designed to track how the investment decisions that agencies have chosen to make and the flexibility to make those decisions adds value to the work of agencies compared to other types of funding. Moreover, the opportunity to meet with agencies enables us to better understand how and why they work the way that they do; how internal and external factors affected strategic decisions processes; and why some decisions were prioritised over others, information that is often not present in the agencies’ annual reports. Understanding these variables allow us to trace how certain investment decisions have resulted in changes to capacity and results. This research and the data and information we collected through discussions with each of the sampled agencies, together with other sources of data, will help us draw conclusions about the performance of the PPA Fund and answer the programme evaluation questions that were set at the beginning. For this reason, our research work with the agencies was focused on understanding how strategic decisions about the use of PPA funding were made across all 12 agencies and the consequences of those decisions. The sample of 12 agencies was agreed with DFID in December 2013. This sample constitutes 28% of the total number of PPA holders and has been purposively selected to be broadly representative of the range of PPA holders. For more information on the research sample, please see Annex D of the Evaluation Strategy.  PPA Evaluation 2014 3 1.3 Organisation profile of Oxfam Oxfam GB’s headquarters in is in Oxford, UK, with major regional and countries offices spread throughout the world. Oxfam has a strong historical relationship with DFID which is reflected in Oxfam’s status as being one of only nine of the current PPA holders to have held a PPA since the funding programme began in 2003. During the current 2011-2014 PPA grant period, Oxfam GB has received £11.2m of combined General and CHASE PPA funding annually which constitutes approximately 3% of Oxfam GB’s total organisational income, and approximately 12% of the unrestricted budget available to its International Division. Oxfam self-identifies as both a “Tier 1” and “Tier 2” that delivers services directly and also through it partners. Oxfam’s multi-sector competence includes a myriad of specialisms across its portfolio of development and humanitarian work. Oxfam GB’s annual turnover places it as a “Large” PPA agency (i.e. Turnover £10m) with a dependency on the PPA funding that is now as “low” (Grant <10% of turnover). Oxfam GB held an additional £23.4m of funding from DFID in 2013-2014, of which approximately £15.1m was used for humanitarian response and £8.2m for Oxfam’s development work. Oxfam used PPA finding as a strategically flexible contribution towards the organisation’s rights-based mandate of alleviating poverty and suffering. 2 Research approach 2.1 Consultation and research process Our research agenda was tailored to primarily meet the needs of the overall Fund-level evaluation, however we attempted to incorporate the learning priorities of Oxfam to the extent that was possible. At all times we attempted to balance the needs of our research against placing an unfair research burden on Oxfam. A timeline of the consultation and research process is summarised below. We first held a familiarisation meeting (with the Team Leader dialling in) at Oxfam House on 23 April 2014. This meeting was used to clarify different elements of the proposed research process, establish lines of communication and start outlining a timetable for the development of research plans. We sent a follow up email to Oxfam with summary points from the meeting on 1 May to which Oxfam provided comments on 15 May. Following the review and analysis of the entire portfolio of 2014 PPA Annual Reports in June, July and August 2014, we shared a list of indicative key points of interest for further research with Oxfam on 29 August. The key points of interest were then discussed through a conference call on 15 September between the research team and Oxfam, the latter who also supplied a document that explained its partnership approach as illustrated by its work with Middle East and Commonwealth of Independent States. A primary research plan was then submitted to Oxfam on 16 September and refined again the following week. The key primary research period spanned two months based on the availability of different stakeholders. Our joint discussions concerning where many of the investments and changes to capacity took place led us to agree to visit Oxfam UK’s headquarters at Oxfam House, Oxford. The key primary research visits took place on 2 and 3 December 2014 by Dr. Neil Macdonald (PPA Evaluation Team Leader) and Peter Mayers (PPA Evaluation Manager). 2.2 Data sources Multiple data sources informed our primary research phase with Oxfam. These included interviews and group discussions with individuals at Oxfam, analysis of previous PPA CHASE and General PPA Annual Reports, logframes, the Independent Progress Review from the 2012 Mid Term Evaluation and review of primary and secondary sources materials that were shared by Oxfam. A full list of people consulted and materials reviewed is included in the Annexes in Section 5. 2.3 Strengths and limitations of approach We found that our research approach satisfied our learning objectives for working with Oxfam. The staff at Oxfam fully cooperated with the research process by granting us open access to personnel and data. Our iterative  PPA Evaluation 2014 4 approach to gathering qualitative data also allowed us to follow emerging lines of inquiry as our understanding of Oxfam GB and the effects of PPA funding improved. Our research approach did have some limitations. The breadth of Oxfam’s work and size made it difficult to delve into some of its more technical work. We also jointly chose to prioritise understanding changes in capacity at the headquarters level (in light of how PPA funding was used) rather than conducting field visits which may have allowed us to see how changes in capacity have led to changes in results. Due to the subjective nature of qualitative data, it was also sometimes difficult to validate or test reliability of data. The data collected from the research visits was, however, used in conjunction with other sources of information, in particular the Annual Reports. 3 Findings 3.1 Use of PPA funding Oxfam works on a one-programme basis, connecting humanitarian, development and campaigning work. The General PPA has been used to support Oxfam’s GB’s foundations of enabling environment and programme quality work. PPA is used solely in international funding, though they do a nominal allocation of funds across the thematic areas (recommended by the IPR) to better understand the use of strategic funds. They commissioned ITAD, who did the IPR, to come back and give guidance on thematic allocation and also Value for Money (VFM). The summary flow diagram below highlights the key factors that informed Oxfam GB’s use of PPA funding; the capacities that were enabled through PPA funding; and the key results from PPA funding.
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks