Analyzing the Causes of the Failure of the Evaluation of the Implementation of Infrastructure Policies in Iran (Field of Research: Industrial Development Policies)

Document Type : Research Paper

Authors

1 Ph.D., Department of Public Administration, Faculty of Management and Accounting, University of Allameh Tabataba'i, Tehran, Iran.

2 Associate Prof., Department of Public Administration, Faculty of Management and Accounting, University of Allameh Tabataba'i, Tehran, Iran.

3 Prof., Department of Public Administration, Faculty of Management and Accounting, University of Allameh Tabataba'i, Tehran, Iran.

4 Prof., Department of Leadership and Human Capital, Faculty of Governance, College of Management , University of Tehran, Tehran, Iran.

Abstract

Objective
Experimental evidence and numerous studies suggest that the evaluation of infrastructure policy implementation in Iran over the past few decades has consistently fallen short of achieving its intended goals. These goals include providing constructive feedback to authorities, enhancing accumulated experience and learning, offering recommendations for policy adjustments or improvements in their implementation, and addressing or eliminating environmental obstacles that hinder policy execution. However, the evaluations conducted have not successfully met these objectives. This research aims to identify the challenges contributing to these shortcomings and seeks to answer the critical question of why these evaluations have been ineffective—referred to as "evaluation failure"—with a particular focus on industrial development policies.
Methods
This research employed a qualitative research method, utilizing thematic analysis to explore the issue in depth. The study involved 48 experts with substantial experience in policy evaluation, selected through targeted and snowball sampling methods. These experts were drawn from four key groups: 1) regulatory organizations (evaluators), 2) universities and research institutes, 3) executive bodies, and 4) policy-making organizations. Through in-depth interviews, data were collected and subsequently analyzed using the thematic analysis method. From these interviews, 564 unique codes were extracted, which formed the basis of the study's findings.
Results
The analysis of the research data revealed five overarching "mega-challenges" within Iran's evaluation system that have significantly contributed to the failure of effective evaluation of infrastructure policies. Of these five mega-challenges, two—namely, "the creation of a necessary institutional framework for effective evaluation" and "weak governance within the evaluation system"—are issues at the national and macro levels, necessitating corrective measures at these broader levels to address them effectively. The remaining three mega-challenges—"challenges related to evaluation methods and processes," "challenges associated with organizational characteristics," and "challenges related to the presentation of evaluation results"—manifest primarily at the organizational level and can potentially be resolved within that context. Additionally, the study identified 17 sub-macro challenges (overarching sub-themes) and 48 more detailed challenges (organizing themes), which are subsets of these mega-challenges. In total, 136 factors (basic themes) were identified, each representing critical elements contributing to the failure of infrastructure policy evaluations in Iran.
Conclusion
The research findings indicate that several significant challenges have undermined the effectiveness of infrastructure policy evaluations in Iran. These challenges have resulted in evaluations that are often quantitatively limited, contradictory, vague, overly general, impractical, and disconnected from the needs of policymakers. Overcoming these challenges is essential to improving the evaluation process, thereby enabling it to achieve its primary objectives: facilitating learning, understanding necessary reforms in policy implementation, and improving the content of approved policies. To address these challenges, the research suggests revising and amending laws and regulations related to evaluation, fostering a culture of evaluation, training evaluation specialists, and creating opportunities for private and non-governmental institutions to engage in evaluation. Additionally, it is recommended to reengineer the structure and context of evaluation organizations to align with necessary features, emphasize local evaluation methods and processes, develop appropriate statistical systems, and ensure the timely and free publication of evaluation results.

Keywords

Main Subjects

References

 
References
Alvani, S.M., Pourezzat, A.A. & Nejabat, A. (2015). Investigating the reasons for the ineffective performance of Iran's policy-making system in the field of poverty alleviation (Case study: The "Poverty Alleviation Bill in the Islamic Republic of Iran"). Public Administration, 7(1), 1-18.  (in Persian)
Arshed, N., Carter, S. & Mason, C. (2014). The ineffectiveness of entrepreneurship policy: is policy formulation to blame? Small Business Economics, 43(3), 639-659.
Ashtarian, K. & Emami Meybodi, R. (2014). Organizing the evaluation of public policies in the Islamic Republic of Iran. Scientific Research Journal of Political Science, 10(2), 5-36. (in Persian)
Azimi, A., Manourian, A., Pourezzat, A.A. & Raghfar, H. (2019). Failure to learn from policy; A critical narrative of privatization experience in Iran. Management Improvement, 13(3), 81-106. (in Persian)
Bemelmans-Videc, M. L., Rist, R. C., Vedung, E., Evert, V. & van der Doelen Frans, C. J. (2017). The sermon: information programs in the public policy process—choice, effects, and evaluation. In Carrots, Sticks & Sermons (pp. 103-128). Routledge.‏
Besharati Rad, Z., Mostahsan, H. & Pasandideh, A. (2009). Evaluation and monitoring model for general policies: A case study of technology policies, under the supervision of Mostafa Mir Saleem, Tehran: Payam Matn. (in Persian)
Blais, É. & Gagné, M. P. (2010). The effect on collisions with injuries of a reduction in traffic citations issued by police officers. Injury prevention, 16(6), 393-397.‏
Braithwaite, J., Churruca, K., Long, J. C., Ellis, L. A. & Herkes, J. (2018). When complexity science meets implementation science: a theoretical and empirical analysis of systems change. BMC medicine, 16(1), 1-14.‌
Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101.
Brynard, P. A. (2009). Policy implementation. Administratio Publica, 17(4), 13-27.‏
Cloete, F. (2009). Evidence-based policy analysis in South Africa: Critical assessment of the emerging government-wide monitoring and evaluation system. Journal of Public Administration, 44(2), 293-311.‏
Cochran, C. E., Mayer, L. C., Carr, T. R., Cayer, N. J. & McKenzie, M. (2015). American public policy: An introduction. Cengage Learning.
DeGroff, A. & Cargo, M. (2009). Policy implementation: Implications for evaluation. New Directions for Evaluation, 2009(124), 47-60.‏
Emami Meybodi, R. & Ashtarian, K. (2012). Designing a public policy evaluation system in the Islamic Republic of Iran. Journal of Political Science Research, 7(2), 7-47. (in Persian)
Emami Meybodi, R. (2013). Policy Evaluation: Conceptual Disorder. Journal of Public Policy, 1(1), 31-48. (in Persian)
Emami Meybodi, R. (2015). Policy Evaluation; Conceptual Disorder? Journal of Public Policy of Tehran University, 1(2), 9-28. (in Persian)
Fink, A. (1999). Principles of Evaluation for Health Program Guides and Policies Researches, (Mehdi Shahvardi and Yousef Shiri, Trans.). Tabriz: Ghazi Jahani. (in Persian)
Gorji Pour, H., Khashaii, V., Islambolchi, A. & Asghari Saremi, A. (2019). Evaluation model for the cultural policy-making process with a qualitative study approach on cultural documents of the Islamic Republic of Iran. Public Administration, 11(1), 47-72. (in Persian)
Hendren, K., Luo, Q. E. & Pandey, S. K. (2018). The state of mixed methods research in public administration and public policy. Public Administration Review, 78(6), 904-916.‏
Hill, M. & Hupe, P. (2014). Implementing public policy: An introduction to the study of operational governance. Sage.‏
Hoseini, S.S. & Rezaei, S. (2010). Determining and Evaluating Policies Supporting Date Producers in Economic Development Programs. Journal of Agricultural Economics and Development Sciences and Industries, 24(1), 33-41. (in Persian)
Hoseini, S.S. & Tarshizi, Mohammad. (2009). Evaluating Wheat Support Policy in Iran. Journal of Agricultural Economics and Development Researches, 40(2), 1-11. (in Persian)
Howlett, M. (2012). The lessons of failure: learning and blame avoidance in public policy-making. International Political Science Review, 33(5), 539-555.‏
Hudson, B., Hunter, D. & Peckham, S. (2019). Policy failure and the policy-implementation gap: can policy support programs help? Policy design and practice, 2(1), 1-14.‏
Jann, W. & Wegrich, K. (2007). Theories of the policy cycle. Handbook of public policy analysis: Theory, politics and methods.
Jann, W. & Wegrich, K. (2017). Theories of the policy cycle. In Handbook of public policy analysis (pp. 69-88). Routledge.‏
Kakhbod, A. (2013). Resource allocation in decentralized systems with strategic agents: an implementation theory approach. Springer Science & Business Media.‏
Khan, A. R. & Khandaker, S. (2016). A critical insight into policy implementation and implementation performance. Viesoji Politika ir Administravimas, 15(4).‏
King, N. & Horrocks, C. (2010). Interviews in qualitative research. London: Sage.
Kvale, S. (1996). Interviews: An Introduction to Qualitative Research Interviewing. Thousand Oaks, CA: Sage Publications.
Leong, C. & Howlett, M. (2022). Policy learning, policy failure, and the mitigation of policy risks: Re-thinking the lessons of policy success and failure. Administration & Society, 54(7), 1379-1401.‏
Lester, J. P. & Stewart, J.R. (2000). The Public Policy Process, Savalan Publications ( Tabari et al., trans.). Savalan Publishing House, 2002. (in Persian)
Maykut, P. & Morehouse, R. (1994). Beginning qualitative research: A Philosophic and practical guide. London: The Falmer Press.
McConnell, A. (2015). What is policy failure? A primer to help navigate the maze. Public Policy and Administration, 30(3-4), 221-242.‏
Mickwitz, P. (2021). Policy evaluation. In Environmental Policy in the EU (pp. 241-258). Routledge.‏
MirMohammadi, S.M. (1999). Comparative study of supervisory systems. Management Studies; Improvement and Transformation, 6(23&24), 63-84.(in Persian)
Mohaghegh Moein, M. H. (2009). Program evaluation basics. Tehran: Moein Empowering Evaluation Institute. (in Persian)
Mohammadi, P., Danaeefard, H. & Zolfagharzadeh, M.M. (2018). Investigating the implications of critical realism in evaluating science policy, technology and innovation. Public Administration, 10(3), 357-386. (in Persian)
Monavvarian, A. (2014). Implementation and evaluation of public policy. (2th ed.). Samt Publishing.
(in Persian)
Mueller, B. (2020). Why public policies fail: Policymaking under complexity. EconomiA, 21(2), 311-323.‏
Munzhedzi, P. H. (2020). Evaluating the efficacy of municipal policy implementation in South Africa: challenges and prospects. African Journal of Governance and Development, 9(1), 89-105.‏
Myerson, R. B. (1991). Game theory: analysis of conflict. Harvard university press.‏
Nagel, S. S. (2018). Public policy evaluation: making super-optimum decisions. Routledge.‏
National Audit Office. (2013). Over-Optimism in Government Projects. London.
Nibler, A. (2014). Technical and commercial evaluation of material supply concepts for a car-assembly hall from an external supply centre.‏
Nilsen, P. (2015). Making sense of implementation theories, models, and frameworks. Implementation science, 30, 53-79.‏
OECD, (2020). How Can Governments Leverage Policy Evaluation to Improve Evidence Informed Policy Making? OECD Publishing, Paris.
Parfrey, T.S. (2002). Handbook of Game Theory. Vol. 3, (Edited by R.J. Aumann & S. Hart). United States of America: Elsevier Science.
Peters, B. G. (2021). Advanced introduction to public policy. Edward Elgar Publishing.‌
Pirayi, Kh. & Fard Doji, S. (2009). Evaluating Iran's Tax Policy from the Perspective of Poverty and Inequality: Has Tax Policy Been Beneficial to the Poor? Quarterly Journal of Economic Researches, 9(2), 19-46. (in Persian)
Pourezzat, A.A. (2021). Designing a policy support system to control policy degradation and misalignment. Public Administration, 13(1), 1-2. (in Persian)
Pourmohammadi, M. R. & Khob Ayand, S. (2001). Review and Evaluation of Housing Policies for Low-Income Urban Groups: A Case Study of Tabriz City. Journal of the Faculty of Literature and Humanities of Tabriz, 44 (180-181), 35-78. (in Persian)
Saidi, Y., Jamalzadeh, A. & Bagheri, A. (2021). Non-judicial oversight over administrative organizations by a similar institution to an ombudsman in Iran. Islamic Law, 18(69), 119-140. (in Persian)
Salimi Far, M., Nourouzi, R. & Motahari, M. (2016). Measuring industrial development and regional development in Khorasan Razavi, South and North provinces. Economic Research Journal, No. 4, 9(4), 175-196. (in Persian)
Scriven, M. (1998). Minimalist theory: The least theory that practice requires. American Journal of Evaluation, 19(1), 57-70. doi:10.1177/109821409801900105.
Shahraki, A. (2014). Evaluating the status of industrial development in the provinces of the country. Quarterly Journal of Strategic Management Studies, 5(17), 19-38. (in Persian)
Solimani Shouraki, H. (2018). Identifying and prioritizing the vulnerabilities of the oversight and inspection system over government agencies from the perspective of managers in Yazd province. Master's thesis at Yazd University. (in Persian)
Spyridaki, N. A., Banaka, S. & Flamos, A. (2016). Evaluating public policy instruments in the Greek building sector. Energy Policy, 88, 528-543.‏
Stafflebim, D. L. (2007). An introduction to evaluation models including standards for program, personnel, and student evaluation. Gholamreza Yadegarzadeh, Arash Bahrami, and Kourosh Parand, Trans.). Tehran: Yadvareh Book. (in Persian)
Stufflebeam, D. L. & Coryn, C. L. (2014). Evaluation theory, models, and applications (Vol. 50). John Wiley & Sons.‏
Timourian, M. & Emami, S.M. (2019). An independent evaluation of the wheat policy goal of the Islamic Republic of Iran. Public Administration, 11(3), 455-480. (in Persian)
Vaezi, R., Sharifzadeh, F. & Mohammadi, M. (2015). Identifying and prioritizing public policy evaluation models. Iranian Journal of Management Sciences, 10(40), 1-22. (in Persian)
Vedung, E. & Bjurulf, S. (2013). A Theory-based Triangulation Approach to Impact Evaluation: The MCET Method Revisited. In Uppsats till Helix International Conference on Innovative Practices in Work, Organization and Regional Development-Problems and Prospects, Linköpings University, Helix VINN Excellence Center, Linköping, 12-14 juni, 2013. Institutet för bostads-och urbanforskning, Uppsala universitet.‏
Wall, D. (2010). Evaluation: improving practice, influencing policy. Understanding medical education: evidence, theory and practice, 336-351.‏
Weiss, C. H. (1988). Reports on topic areas: Evaluation for decisions: Is anybody there? Does anybody care? Evaluation practice, 9(1), 5-19.‏
Wholey, J. S., Scanlon, J. W., Duffy, H. G., Fukumoto, J. S. & Vogt, L. M. (1970). Federal evaluation policy: Analyzing the effects of public programs. Washington, DC: Urban Institute.‏
Wollmann, H. (2017). Policy evaluation and evaluation research. In Handbook of public policy analysis (pp. 419-428). Routledge.‏
Zhang, G., Zeller, N., Griffith, R., Metcalf, D., Williams, J., Shea, C. & Misulis, K. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement, 15(4), 57-84.
Journal of Public Administration
Volume 17, Issue 1
2025
Pages 13-43

Files

Share

How to cite

Statistics