World Scientific
  • Search
    Quick Search in Journals
    Access type:
    Quick Search anywhere
    Access type:
    Advanced Search
  • My Cart
Skip main navigation
Close Drawer MenuOpen Drawer Menu

Cookies Notification

We use cookies on this site to enhance your user experience. By continuing to browse the site, you consent to the use of our cookies. Learn More
×
Our website is made possible by displaying certain online content using javascript.
In order to view the full content, please disable your ad blocker or whitelist our website www.worldscientific.com.

System Upgrade on Tue, Oct 25th, 2022 at 2am (EDT)

Existing users will be able to log into the site and access content. However, E-commerce and registration of new users may not be available for up to 12 hours.
For online purchase, please visit us again. Contact us at [email protected] for any enquiries.
Research PapersOpen Access

Impacts Assessments without True Baselines: Assessing the Relative Effects of Training on the Performance of Water User Associations in Southern Tajikistan

    https://doi.org/10.1142/S2382624X18500078Cited by:5
    Previous
    Next
    This article is part of the issue:
    Special Issue on The Institutional Economics of Water

    Abstract

    Conducting rigorous evaluations of whether the process of creating new institutions affects their performance of mandated duties presents several challenges. Not only is assignment to process often not random, but when the process of creating new institutions starts, outcomes and other performance-influencing covariates are not measurable because the yet-to-be created institutions are not functioning at baseline. This paper compares the performance of 74 ‘treated’ water user associations (WUAs) in Tajikistan that were created using a longer training process with 67 ‘control’ WUAs that were created using shorter training, to assess the impact of training on WUA performance of mandated duties. First, propensity scores were constructed to estimate the probabilities of being ‘treated’ by treatment status. These results guided the application of the difference-in-difference technique with right-hand side covariates in a context where field measures of outcomes and other performance-influencing covariates were made after the new institutions were created and functioning. The first measures were taken within 12–18 months of the new institutions being functional and the second measures were taken 24 months after the first. This choice of methods introduces a bias due to measurement error causing an underestimate of the treatment effects, while controlling for biases due to time-invariant and time-varying unobservables. An alternative method that only compared the differences in outcomes at a single point in time after the new institutions were created would have provided an inaccurate estimate of the effects of the intervention. This is a context in which methods such as synthetic controls are impossible to employ due to the nature of the intervention, other macroeconomic structural changes, and severe data restrictions. The methodology employed here generates evidence that, while biased toward generating an underestimate of effect, can still be useful and informative for policy and management purposes, and for evaluating the impact of process on the functioning of new institutions in transition settings.

    References

    • Abadie, A, A Diamond and J Hainmueller [2010] Synthetic control methods for comparative case studies: Estimating the effect of California’s tobacco control program. Journal of the American Statistical Association, 105, 493–505. Crossref, Google Scholar
    • Abadie, A and J Gardeazabal [2003] The economic costs of conflict: A case study of the Basque country. American Economic Review, 93, 113–132. Crossref, Google Scholar
    • Asian Development Bank (ADB) (2012). Tajikistan: Irrigation Rehabilitation Project completion report. Validation report, Independent Evaluation Department. Manila: ADB. Google Scholar
    • Balasubramanya, S, M-C Buisson, P Saikia, K MacDonald, S Aslamy, T Horbulyk, C Hannah, M Yakubov and A Platonov (2016). Impact of water user associations on water and land productivity, equity and food security in Tajikistan. Baseline report, prepared for the United States Agency for International Development (USAID), Colombo: International Water Management Institute, pp. 1–127. https://agrilinks.org/sites/default/files/resource/files/Feed%20the%20Future%20Tajikistan%20Water%20User%20Associations%20Impact%20Evaluation%20-%20B.pdf (last accessed 15 June 2017). Google Scholar
    • Barcellini, F, L Prost and C Marianne [2015] Designers’ and users’ roles in participatory design: What is actually co-designed by participants? Applied Ergonomics, 50, 31–40. Crossref, Google Scholar
    • Batt, HA and GP Merkley [2010] Water management and user association analysis for irrigation improvement in Egypt. Irrigation and Drainage, 59, 150–160. Google Scholar
    • Çakmak, B, M Beyyribey and S Kodal [2004] Irrigation water pricing in water user associations, Turkey. Water Resource Development, 20, 113–124. Crossref, Google Scholar
    • Denyer Willis, G and M Mota Prado [2014] Process and pattern in institutional reforms: A case study of the Police Pacifying Units (UPPs) in Brazil. World Development, 64, 232–242. Crossref, Google Scholar
    • Fischer, ME, B Irlenbusch and A Sadrieh [2004] An intergenerational common pool resource experiment. Journal of Environmental Science and Management, 48, 811–836. Google Scholar
    • Food and Agriculture Organization of the United Nations (FAO) (2012). AQUASTAT: Tajikistan. http://www.fao.org/nr/water/aquastat/countries_regions/TJK/ (last accessed 9 January 2017). Google Scholar
    • Gunchinmaa, T and M Yakubov [2009] Institutions and transition: Does a better institutional environment make water users associations more effective in Central Asia? Water Policy, 12 (2), 165–185. Crossref, Google Scholar
    • Hamidov, A, A Thiel and D Zikos [2015] Institutional design in transformation: A comparative study of local irrigation governance in Uzbekistan. Environmental Science and Policy, 53, 175–191. Crossref, Google Scholar
    • Hornidge, AK, L Oberkircher and A Kudryavtseva [2013] Boundary management and the discursive sphere — Negotiating realities in Khorezm, Uzbekistan. Geoforum, 45, 266–274. Crossref, Google Scholar
    • Huang, Q, J Wang, KW Easter and S Rozelle [2010] Empirical assessment of water management institutions in Northern China. Agricultural Water Management, 98, 361–369. Crossref, Google Scholar
    • Johnson, SH and J Stoutjesdijk [2008] WUA training and support in the Kyrgyz Republic. Irrigation and Drainage, 57, 311–321. Crossref, Google Scholar
    • Kazbekov, J, I Abdullaev, H Manthrithilake, A Qureshi and K Jumaboev [2009] Evaluating planning and delivery performance of water user associations (WUAs) in Osh Province, Kyrgyzstan. Agricultural Water Management, 96, 1259–1267. Crossref, Google Scholar
    • Matveeva, A (2009). The perils of emerging statehood: Civil war and state reconstruction in Tajikistan. Working paper, Number 46, Series 2. London: Crisis States Research Centre, pp. 1–59. Google Scholar
    • Meinzen-Dick, R, KV Raju and A Gulati [2002] What affects organization and collective action for managing resources? Evidence from canal irrigation systems in India. World Development, 30 (4), 649–666. Crossref, Google Scholar
    • Mukhtarov, F, S Fox, N Mukhamedova and K Wegerich [2015] Interactive institutional design and contextual relevance: Water user groups in Turkey, Azerbaijan and Uzbekistan. Environmental Science and Policy, 53, 206–214. Crossref, Google Scholar
    • Mustafa, D, A Altz-Stamm and L Mapstone Scott [2016] Water user associations and the politics of water in Jordan. World Development, 79, 164–176. Crossref, Google Scholar
    • Nagrah, A, AM Chaudhry and M Giordano [2016] Collective action in decentralized irrigation systems: Evidence from Pakistan. World Development, 84, 282–298. Crossref, Google Scholar
    • O’Hara, SL [2000] Lessons from the past: Water management in Central Asia. Water Policy, 2, 365–384. Crossref, Google Scholar
    • Ostrom, E [1990] Governing the Commons: The Evolution of Institutions for Collective Action, pp. 1–294. Cambridge, UK: Cambridge University Press. Crossref, Google Scholar
    • Republic of Tajikistan [2006] Water user association law. Law of the Republic of Tajikistan, pp. 1–12. Google Scholar
    • Ricks, JI [2016] Building participatory organizations for common pool resource management: Water user group promotion in Indonesia. World Development, 77, 34–47. Crossref, Google Scholar
    • Rosenbaum, PR and DJ Rubin [1985] Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. The American Statistician, 39, 33–38. Crossref, Google Scholar
    • Rubin, DJ [2001] Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2, 169–188. Crossref, Google Scholar
    • Sehring, J [2007] Irrigation reform in Kyrgyzstan and Tajikistan. Irrigation and Drainage Systems, 21, 277–290. Crossref, Google Scholar
    • Shahriari, H (2009). Activities, water and gender in Tajikistan’s rural sector: A social assessment of Konibodom, Bobojon Ghafurov, and Yovon. Social Development Department, Europe and Central Asia series. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/364511468117242593/pdf/514450WP0TJ0So10Box342028B01PUBLIC1.pdf (last accessed 15 June 2017). Google Scholar
    • Skaperdas, S [2001] The political economy of organized crime: Providing protection when the state does not. Economics of Governance, 2, 173–202. Crossref, Google Scholar
    • Smith, JL [2008] A critical appreciation of the ‘bottom-up’ approach to sustainable water management: Embracing complexity rather than desirability. Local Environment, 13 (4), 353–366. Crossref, Google Scholar
    • Tellez Foster, E, A Rapoport and A Dinar (2016). Alternative policies to manage electricity subsidies for groundwater extraction: A field study in Mexico. UCR SPP Working Paper Series WP#16-05. Riverside, California: School of Public Policy, University of California-Riverside. Google Scholar
    • Tellez Foster, E, A Rapoport and A Dinar [2017] Groundwater and electricity consumption under alternative subsidies: Evidence from laboratory experiments. Journal of Behavioral and Experimental Economics, 68, 41–52. Crossref, Google Scholar
    • Theesfeld, I [2004] Constraints on collective action in a transitional economy: The case of Bulgaria’s irrigation sector. World Development, 32 (2), 251–271. Crossref, Google Scholar
    • Thiel, A, F Mukhtarov and D Zikos [2015] Crafting or designing? Science and politics for purposeful institutional change in social-ecological systems. Environmental Science and Policy, 53, 81–86. Crossref, Google Scholar
    • United Nations (2015). Tajikistan. http://data.un.org/CountryProfile.aspx?crName=TAJIKISTAN (last accessed 22 June 2017). Google Scholar
    • United States Agency for International Development (USAID) [2014] USAID Family Farming Program — Quarterly Report: Year Four — First Quarter. Unpublished document, pp. 1–46. Washington, DC: USAID. Google Scholar
    • Wegerich, K (2000). Water user associations in Uzbekistan and Kyrgyzstan: Study on conditions for sustainable development. Occasional paper — Water Issues Study Group, London: School of Oriental and African Studies, University of London, Number 32, pp. 1–23. Google Scholar
    • Woolridge, J [2002] Econometric Analysis of Cross Section and Panel Data. Cambridge, MA: The MIT Press. Google Scholar
    • World Bank (2015). Tajikistan. http://data.worldbank.org/country/tajikistan (last accessed 22 June 2017). Google Scholar
    • Yakubov, M [2011] Assessing irrigation performance from the farmers’ perspective: A qualitative study. Irrigation and Drainage, 61 (3), 316–329. Crossref, Google Scholar
    • Yap-Salinas, LH (1994). Converging factors in the successful transfer of irrigation management responsibilities to water users’ associations in the Dominican Republic. Paper presented at the Int. Conf. Irrigation Management Transfer, 20–24 September, Wuhan, China, pp. 117–127. Google Scholar
    • Zhou, Q [2013] Decentralized irrigation in China: An institutional analysis. Policy and Society, 32, 77–88. Crossref, Google Scholar