IDEAS home Printed from https://ideas.repec.org/a/inm/ormnsc/v67y2021i8p4982-5010.html

A Theory of Quasi-Experimental Evaluation of School Quality

Author

Listed:
  • Yusuke Narita

    (Department of Economics and Cowles Foundation, Yale University, New Haven, Connecticut 06520)

Abstract

Many centralized school admissions systems use lotteries to ration limited seats at oversubscribed schools. The resulting random assignment is used by empirical researchers to identify the effects of schools on outcomes like test scores. I first find that the two most popular empirical research designs may not successfully extract a random assignment of applicants to schools. When are the research designs able to overcome this problem? I show the following main results for a class of data-generating mechanisms containing those used in practice: The first-choice research design extracts a random assignment under a mechanism if the mechanism is strategy-proof for schools. In contrast, the other qualification instrument research design does not necessarily extract a random assignment under any mechanism. The former research design is therefore more compelling than the latter. Many applications of the two research designs need some implicit assumption, such as large-sample approximately random assignment, to justify their empirical strategy.

Suggested Citation

  • Yusuke Narita, 2021. "A Theory of Quasi-Experimental Evaluation of School Quality," Management Science, INFORMS, vol. 67(8), pages 4982-5010, August.
    • Handle: RePEc:inm:ormnsc:v:67:y:2021:i:8:p:4982-5010
    DOI: 10.1287/mnsc.2020.3742
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/mnsc.2020.3742
    Download Restriction: no
    File URL: https://libkey.io/10.1287/mnsc.2020.3742?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jackson, C. Kirabo, 2012. "Single-sex schools, student achievement, and course selection: Evidence from rule-based student assignments in Trinidad and Tobago," Journal of Public Economics, Elsevier, vol. 96(1), pages 173-187.
    2. Adrienne M. Lucas & Isaac M. Mbiti, 2014. "Effects of School Quality on Student Achievement: Discontinuity Evidence from Kenya," American Economic Journal: Applied Economics, American Economic Association, vol. 6(3), pages 234-263, July.
    3. Caterina Calsamiglia & Chao Fu & Maia Güell, 2014. "Structural Estimation of a Model of School Choices: the Boston Mechanism vs. Its Alternatives," Working Papers 2014-21, FEDEA.
    4. David J. Deming, 2011. "Better Schools, Less Crime?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 2063-2115.
    5. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    6. Howard S. Bloom & Rebecca Unterman, 2014. "Can Small High Schools of Choice Improve Educational Prospects for Disadvantaged Students?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(2), pages 290-319, March.
    7. Hatfield, John William & Kojima, Fuhito & Narita, Yusuke, 2016. "Improving schools through school choice: A market design approach," Journal of Economic Theory, Elsevier, vol. 166(C), pages 186-211.
    8. Yusuke Narita & Kohei Yata, 2021. "Algorithm is Experiment: Machine Learning, Market Design, and Policy Eligibility Rules," Working Papers 2021-022, Human Capital and Economic Opportunity Working Group.
    9. Adam J. Kapor & Christopher A. Neilson & Seth D. Zimmerman, 2020. "Heterogeneous Beliefs and School Choice Mechanisms," American Economic Review, American Economic Association, vol. 110(5), pages 1274-1315, May.
    10. Fort, Margherita & Zanella, Giulio, 2019. "Cognitive and non-cognitive costs of daycare 0–2 for children in advantaged families," CEPR Discussion Papers 11120, Centre for Economic Policy Research.
    11. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    12. Chung-Piaw Teo & Jay Sethuraman & Wee-Peng Tan, 2001. "Gale-Shapley Stable Marriage Problem Revisited: Strategic Issues and Applications," Management Science, INFORMS, vol. 47(9), pages 1252-1267, September.
    13. Shapley, Lloyd & Scarf, Herbert, 1974. "On cores and indivisibility," Journal of Mathematical Economics, Elsevier, vol. 1(1), pages 23-37, March.
    14. Ivan A. Canay & Joseph P. Romano & Azeem M. Shaikh, 2017. "Randomization Tests Under an Approximate Symmetry Assumption," Econometrica, Econometric Society, vol. 85, pages 1013-1030, May.
    15. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769, December.
    16. Cristian Pop-Eleches & Miguel Urquiola, 2013. "Going to a Better School: Effects and Behavioral Responses," American Economic Review, American Economic Association, vol. 103(4), pages 1289-1324, June.
    17. , A. & ,, 2011. "Lotteries in student assignment: An equivalence result," Theoretical Economics, Econometric Society, vol. 6(1), January.
    18. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2012. "The Effect of School Choice on Intrinsic Motivation and Academic Outcomes," Working Papers 2012-3, Princeton University. Economics Department..
    19. Azevedo, Eduardo M., 2014. "Imperfect competition in two-sided matching markets," Games and Economic Behavior, Elsevier, vol. 83(C), pages 207-223.
    20. Justine S. Hastings & Christopher A. Neilson & Seth D. Zimmerman, 2013. "Are Some Degrees Worth More than Others? Evidence from college admission cutoffs in Chile," NBER Working Papers 19241, National Bureau of Economic Research, Inc.
    21. Eduardo M. Azevedo & Jacob D. Leshno, 2016. "A Supply and Demand Framework for Two-Sided Matching Markets," Journal of Political Economy, University of Chicago Press, vol. 124(5), pages 1235-1268.
    22. Roth, Alvin E & Sotomayor, Marilda, 1989. "The College Admissions Problem Revisited," Econometrica, Econometric Society, vol. 57(3), pages 559-570, May.
    23. C. Kirabo Jackson, 2010. "Do Students Benefit from Attending Better Schools? Evidence from Rule-based Student Assignments in Trinidad and Tobago," Economic Journal, Royal Economic Society, vol. 120(549), pages 1399-1429, December.
    24. Fuhito Kojima & Parag A. Pathak, 2009. "Incentives and Stability in Large Two-Sided Matching Markets," American Economic Review, American Economic Association, vol. 99(3), pages 608-627, June.
    25. Gary Chamberlain, 1980. "Analysis of Covariance with Qualitative Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 47(1), pages 225-238.
    26. David J. Deming & Justine S. Hastings & Thomas J. Kane & Douglas O. Staiger, 2014. "School Choice, School Quality, and Postsecondary Attainment," American Economic Review, American Economic Association, vol. 104(3), pages 991-1013, March.
    27. Elliott Peranson & Alvin E. Roth, 1999. "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design," American Economic Review, American Economic Association, vol. 89(4), pages 748-780, September.
    28. Alexander Shapiro & Jos Berge, 2002. "Statistical inference of minimum rank factor analysis," Psychometrika, Springer;The Psychometric Society, vol. 67(1), pages 79-94, March.
    29. Fort, Margherita & Ichino, Andrea & Zanella, Giulio, 2016. "Cognitive and Non-Cognitive Costs of Daycare 0–2 for Girls," IZA Discussion Papers 9756, IZA Network @ LISER.
    30. Yusuke Narita, 2021. "Incorporating ethics and welfare into randomized experiments," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(1), pages 2008740118-, January.
    31. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.
    32. Yan Chen & Onur Kesten, 2017. "Chinese College Admissions and School Choice Reforms: A Theoretical Analysis," Journal of Political Economy, University of Chicago Press, vol. 125(1), pages 99-139.
    33. He, Yinghua, 2012. "Gaming the Boston School Choice Mechanism in Beijing," TSE Working Papers 12-345, Toulouse School of Economics (TSE).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yusuke Narita, 2020. "A Theory of Quasi-Experimental Evaluation of School Quality," Working Papers 2020-085, Human Capital and Economic Opportunity Working Group.
    2. Yusuki Narita, 2016. "(Non)Randomization: A Theory of Quasi-Experimental Evaluation of School Quality," Cowles Foundation Discussion Papers 2056, Cowles Foundation for Research in Economics, Yale University.
    3. Atila Abdulkadiroğlu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Research Design Meets Market Design: Using Centralized Assignment for Impact Evaluation," Econometrica, Econometric Society, vol. 85, pages 1373-1432, September.
    4. Atı̇la Abdulkadı̇roğlu & Joshua D. Angrist & Yusuke Narita & Parag Pathak, 2022. "Breaking Ties: Regression Discontinuity Design Meets Market Design," Econometrica, Econometric Society, vol. 90(1), pages 117-151, January.
    5. Atila Abdulkadiroglu & Joshua D. Angrist & Yusuke Narita & Parag A. Pathak, 2017. "Impact Evaluation in Matching Markets with General Tie-Breaking," NBER Working Papers 24172, National Bureau of Economic Research, Inc.
    6. Chen, Jiafeng, 2026. "Nonparametric treatment effect identification in school choice," Journal of Econometrics, Elsevier, vol. 253(C).
    7. Gabrielle Fack & Julien Grenet & Yinghua He, 2019. "Beyond Truth-Telling: Preference Estimation with Centralized School Choice and College Admissions," American Economic Review, American Economic Association, vol. 109(4), pages 1486-1529, April.
    8. Cao, Yuan, 2020. "Centralized assignment mechanisms and assortative matching: Evidence from Chinese universities," Journal of Economic Behavior & Organization, Elsevier, vol. 177(C), pages 255-276.
    9. Li, Mengling, 2020. "Ties matter: Improving efficiency in course allocation by allowing ties," Journal of Economic Behavior & Organization, Elsevier, vol. 178(C), pages 354-384.
    10. Schwartz, Jacob & Song, Kyungchul, 2024. "The law of large numbers for large stable matchings," Journal of Econometrics, Elsevier, vol. 241(1).
    11. Atila Abdulkadiroglu & Tommy Andersson, 2022. "School Choice," NBER Working Papers 29822, National Bureau of Economic Research, Inc.
    12. Ha, Wei & Kang, Le & Song, Yang, 2020. "College matching mechanisms and matching stability: Evidence from a natural experiment in China," Journal of Economic Behavior & Organization, Elsevier, vol. 175(C), pages 206-226.
    13. Artemov, Georgy, 2021. "Assignment mechanisms: Common preferences and information acquisition," Journal of Economic Theory, Elsevier, vol. 198(C).
    14. Thierry Magnac, 2018. "Quels étudiants pour quelles universités ? Analyses empiriques de mécanismes d’allocation centralisée," Revue économique, Presses de Sciences-Po, vol. 69(5), pages 683-708.
    15. Aaron L. Bodoh-Creed, 2020. "Optimizing for Distributional Goals in School Choice Problems," Management Science, INFORMS, vol. 66(8), pages 3657-3676, August.
    16. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    17. Marco Ovidi, 2022. "Parents Know Better: Sorting on Match Effects in Primary School," DISCE - Working Papers del Dipartimento di Economia e Finanza def121, Università Cattolica del Sacro Cuore, Dipartimenti e Istituti di Scienze Economiche (DISCE).
    18. Eduardo Duque & Juan Pablo Torres-Martinez, 2022. "The Strong Effects of Weak Externalities on School Choice," Working Papers wp542, University of Chile, Department of Economics.
    19. Alvin Roth, 2008. "Deferred acceptance algorithms: history, theory, practice, and open questions," International Journal of Game Theory, Springer;Game Theory Society, vol. 36(3), pages 537-569, March.
    20. Nikhil Agarwal & Paulo Somaini, 2018. "Demand Analysis Using Strategic Reports: An Application to a School Choice Mechanism," Econometrica, Econometric Society, vol. 86(2), pages 391-444, March.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:67:y:2021:i:8:p:4982-5010. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.