The purpose of the proposed project is twofold: First, it will clarify and disseminate practical applications of research strategies that will convey the design benefits of randomized field trials; second, it will sharpen concepts that form the theoretical basis for randomized field trials. Key underlying concepts include the nature and limits of inferences that can be drawn from randomized field trials, and warranted reasoning about causal relations between educational interventions and learning outcomes. Project outcomes will add value to STEM education projects within the broad portfolio of the National Science Foundation, including projects within the Math-Science Partnership (MSP) program.

Gains in practical and conceptual knowledge related to randomized field trials in education will be applicable in the short-term to currently funded projects, to prospective projects submitted in response to current program announcements, and to the reformulation of future program announcements by NSF leadership. This project will focus on organizing consensus knowledge about the ideal role of randomized treatments within a research and development portfolio in STEM education, with the explicit purpose of using those gains in knowledge to improving learning outcomes. Moreover, because the application of randomized field methodologies is in an early stage in educational research in comparison to medicine and economics, the intellectual products of the proposed project will advance this complex subfield, not simply organize the debates and arguments attendant to the many important issues related to conducting randomized field trials in education.

The conceptual and practical difficulties of designing and using randomized field trials in STEM education research, and in educational research more broadly, shows that the subject requires the sustained attention of scholars. The conceptualization and implementation of experimental methodologies is highly germane to policy and funding decisions, and even more relevant to the priorities of the STEM education researcher. Inferences from data about program effects bear strongly on the intellectual merit and broader impacts of work supported by the National Science Foundation and other agencies, governmental and private, that support research designed to enhance the power and efficacy of educational interventions. Questions about research methodology in STEM education must address what claims are warranted from data (bearing on intellectual merit) and what research designs are practicable and in the best interests of the collaborating educators and their students (broader impacts). The proposed project addresses the NSF-wide criteria of intellectual merits and broader impacts by defining these criteria with respect to research design methodology. In the longer term, the project will illuminate for prospective PIs, NSF program directors, and funded investigators the numerous ways in which the intellectual merit and broader impacts of NSF-sponsored projects can be enhanced.