Policy, Program and Project Evaluation: A Toolkit for Economic Analysis in a Changing World

Preface
Acknowledgements
Contents
List of Figures
List of Tables
List of Boxes
Notes on Contributors
Chapter 1: Introduction
Reference
Chapter 2: A Primer on Public Sector Evaluations
Introduction
Leading Approaches to Evaluation
Cost-Benefit Analysis
Cost-Effectiveness Analysis
Social Marginal Cost of Funds Analysis
Data Envelopment Analysis
Multiple-Criteria Evaluation (MCE)/Multiple-Objectives Evaluation (MOE)
Diagnostic Evaluation
Process Evaluation
Outcome Evaluation
Outcome Evaluation Designs
Randomized Experimental Designs
Non-experimental Designs
Matched Control Designs
Difference-in-Differences and Fixed Effects Regression
Reflexive Control Designs
Regression Discontinuity Design
Instrumental Variables Approach
Newer Approaches to Evaluation
The Iron Triangle of Project Management and Evaluation
Alternative Service Delivery Framework for Evaluation
Result-Oriented Management and Evaluation (ROME)
Theory-Based Evaluation
Conclusion
Annex: An Example of a Multi-Criteria Evaluation Approach—The Practice by the World Bank Operations Evaluation Department/the Independent Evaluation Group
Relevance of Objectives
Efficacy
Efficiency (by OED; the IEG Dropped This Criterion)
Sustainability (by OED; the IEG Dropped This Criterion)
Result (New Criterion by the IEG)
Outcome/Effectiveness
Institutional Development Impact (IDI; by the OED Only)
The Bank Performance
The Borrower Performance (by the OED Only)
References
Chapter 3: Economic Evaluation of Projects
Principles of Valuation
The Decision Rule
The Present Value Criterion
Alternative PV Formulations I: The Benefit-Cost Ratio
Alternative PV Formulations II: The Internal Rate of Return
Capital Budgeting
The Treatment of Inflation
Terminal Value
Choice of a Numeraire
Present Versus Future Consumption
Foreign Exchange
Public Financing
Investment Relative to Consumption
Distributive Equity Considerations
Cost-Effectiveness Versus Benefit-Cost Analysis
Sensitivity Analysis
Valuing Inputs and Outputs
Market Inputs and Outputs
The Shadow Price of Foreign Exchange
The Shadow Wage Rate
Special Problems with Capital Inputs
Intangibles and Non-marketed Inputs and Outputs
Value of Time Saved
Value of Reduced Risk of Death
Costs of Environmental Pollution
Survey Techniques: Contingent Valuation
Subsidies
Indirect Benefits and Costs
The Social Discount Rate
Heterogeneous Household Interest Rates
Capital Market Imperfections
Externalities in Capital Markets
Other Arguments: Risk and Distributive Weights
The Numeraire
Oppurtunity Cost of Finanacing
The Marginal Cost of Public Funds
The Opportunity Cost of Borrowed Funds
Risk and Uncertainty
Distributive Weights
Concluding Remarks
References
Chapter 4: The Marginal Cost of Public Funds: Concept, Measurement, and Applications
Introduction
Conceptual Foundations
Measuring the MCF
Formula-Based Calculation
CGE Computations of the MCFs
Econometric Models
Using the MCF to Evaluate Tax Reforms and Public Expenditure Programs
Evaluating Tax Reforms
Designing Intergovernmental Infrastructure Grants
Concluding Remark
References
Chapter 5: Theory-Based Evaluations: Guidelines for Case Studies in Program and Project Evaluation
Introduction
Case A: Privatization Programs
Program Theory
Decrease in the Supply of Public Finance
Development of a New Private Sector Constituency
The Analysis
Decrease in the Supply of Public Finance
Development of a New Class of Private Business Owners
The Data
Decrease in the Supply of Public Finance
Development of a New Private Sector Constituency
Case B: Judicial and Legal Reforms
Program Theory
Strengthening Enforcement of Judicial Decisions
Transparency of the Legal System
Independence of the Judiciary
The Analysis
Strengthening of Enforcement
Strengthening Independence of the Judiciary
Transparency of the Legal System
The Data
Strengthening Enforcement
Strengthening Independence of the Judiciary
Transparency of the Legal System
Case C: Civil Service Reforms
Program Theory
Improved Compensation of Civil Service Workers
Improved Management of Service Delivery
The Analysis
Improved Compensation of Civil Service Workers
Improved Management of Service Delivery
The Data
Improved Compensation in Civil Service
Improved Management of Service Delivery
Case D: Trade Liberalization
Program Theory
Elimination of Import License Requirements
Simplifying and Unifying the Tariff Code
Lowering or Eliminating Import Tariffs
The Analysis
Elimination of Import Licenses
Simplification of the Tariff Code
The Data
Elimination of Import Permits
Delays and Restrictions in Foreign Trade
Contacts Between Importers and Public Officials
Opportunities to Extract Bribes by Public Officials
Corruption in Business Service
Degree of Discretionary Power of Customs Officials/Contacts Between Importers and Customs Officers
Case E: Tax Administration Reforms
Program Theory
Simplification of the Tax System
Self-Assessment of Tax Obligation and Self-Payment of Taxes
Bonus Incentives for Tax Inspectors
The Analysis
Simplification of the Tax System/Self-Assessment of Tax Obligation
Bonus Incentives for Tax Inspectors
The Data
Simplification of the Tax System
Self-Assessment and Self-Payment of Tax
Bonus Incentives for Tax Inspectors
Case F: Direct Anticorruption Activities (Anitcorruption Agencies, the Office of the Ombudsman, Transparency Rules, and Decentralization)
Program Theory
Anticorruption Agencies
The Office of the Ombudsman
Transparency Rules
Decentralization
The Analysis
Anticorruption Agency
The Office of the Ombudsman
Transparency Rules
Decentralization
The Data
Anticorruption Agency
Basic Functions of the Agency
Intermediate Links in the Chain of Causal Assumptions
Behavior of Public Officials, Opportunities to Extract Bribes
The Office of the Ombudsman
Basic Functions of the Office
Intermediate Links in the Chain of Causal Assumptions
Opportunities to Extract Bribes
Transparency Rules
Decentralization
References
Chapter 6: A Framework for Evaluating Anti-Corruption Policies and Programs
A Simple Evaluation Framework
Corruption in the Presence of Altruism
Adapting World Bank Evaluation Methodology to the Evaluation of Anti-Corruption Programs
Relevance
Efficacy
Efficiency
Sustainability
Conclusion
Annex: Framework for Evaluation of the Relevance of Anti-Corruption Policies—Principal-Agent Theory and New Institutional Economics
Moral Hazard Issues
Institutional Analysis of Corruption
References
Chapter 7: Evaluating External Analytical Advice on Budgetary Institutions and Allocations
Introduction
Methodology
Evaluation of the Quality and Timeliness of Alternate Forms of PERs
(a) Rigor, Consistency, and Clarity
(b) Internalization of Findings of Previous Studies
(c) Evidence of Consultations with Relevant Levels of Government and Stakeholders
(d) Selection of Issues Addressed and Depth of Such Analyses
(e) Recognition of Political, Institutional, Informational, and Economic Imperatives of the Client
Overall Quality and Timeliness
Evaluation of the Impact of PER Instruments
(a) Aid-Coordination Efforts by the Development Assistance Community
(b) Borrower Policies
(c) Cost-Efficiency
Overall Impact
Annex: Getting Government Right—The Canadian Approach to PER
References
Chapter 8: Inter-Sectoral Allocation Choices
Introduction
Rationales for Government Intervention in the Allocation of Resources: A Framework for Analysis
Insufficient Competition
Incomplete Markets
Public Goods
Asymmetric Information
Redistribution
Section Summary
Inter-Sectoral Resource Allocation: Public Sector Administration
Capacity and Efficiency of the Administration
Size of the Civil Service Wage Bill
Monitoring Expenditure Effectiveness
Private Sector Provision of Goods
User Fees
Section Summary
Sector-Specific Issues in the Allocation of Government Resources
Law and Security
Military Spending
Infrastructure Investment
Transportation Infrastructure
Operations and Maintenance
Education
Primary Versus Tertiary Education
The Health Sector
Redistribution
Regulation
Financial Markets
State-Owned Enterprises
Industrial Subsidies
Conclusion
References
Chapter 9: Evaluation of Decentralization Programs
Introduction
The Basic Methodology
Components of Decentralization
Government and Political Institutions
Responsibilities and Powers
Resources
Taxes and Other Sources of Own Revenue
Intergovernmental Transfers
Deficits and Debt
Administrative Capacity
Management Authority
Accountability
Beyond Government
Concluding Note
Steps in Evaluating Decentralization Projects
Environment
Action
Outcomes
Assessment
Relevance
Efficacy
Sustainability
Efficiency
Institutional Development
Bank Performance
Country (Borrower) Performance
Lessons
Methods of Analysis
Comparing Evaluations Across Many Projects: Verbal and Quantitative Approaches
The Verbal Approach
A Quantitative Approach
Desk and Country Analyses
Desk Studies
Country Studies
A Possible Extension
Selection of the Projects for Analysis
In-Depth Desk Study
Country Studies
Bringing It All Together
Summary and Conclusion
Appendix: Components of Decentralization and Factors Affecting the Potential Success of Decentralization
References
Chapter 10: Evaluating the Conditionality of External Development Assistance Programs
Introduction
Rationale for the Conditionality
The Conditionality and the Donor–Recipient Government Relationship
Donor Influence Over Political Costs
Country Administrators’ Influence Over Donor Conditions
Conceptual Perspectives
Program Design and the Conditionality: Conceptual Perspectives
Game Theory
Placing Loan (Grant) Conditions in a Game Theory Framework
A Numerical Example
A Three-Player Example
Improvements in Donor-Imposed Conditions Suggested by Game Theory
Limitations of the Examples Presented Here
Game Theory Conclusion
Public Choice Perspectives
Limitations of the Approach
Fiscal Federalism Perspective
Limitations of the Approach
New Public Management Perspectives
Limitations of the Approach
Political Economy Perspectives
Limitations of the Approach
New Institutional Economics Perspectives
Limitations of the Approach
A Synthesis of Conceptual Perspectives
Evolving Institutional Perspectives on the Conditionality
International Development Assistance: Evolving Perspectives on the Conditionality of Assistance
Evolving Landscape on the Conditionality of Development Assistance
Loan Conditions
Grant Conditions
Emerging Consensus on the New Model of Conditionality and Current Practice
Outstanding Issues and Views
Policy-Based Lending
Minimum International Standards
Mutual Accountability
Country Ownership and Conditionality: Some Tensions
Current State of Partnership for Development Assistance
Lessons on the Conditionality of Development Assistance
References
Author Index
Subject Index