Prof. Mark Constas, education, thinks a group of scholars has been overlooked in his field: those committed to using scientific methods to advance the study of education. And with a recent $750,000 grant from the U.S. Department of Education, he is doing something about it.
Constas says he will use federal funds to establish the Society for Research on Education Effectiveness (SREE). Among the society’s goals are publishing a peer-reviewed quarterly journal, Journal of Research on Education Effectiveness; compiling a handbook highlighting recent education research; and holding an annual conference starting in late 2007.
But to Constas, the most valuable product of this grant, on which he is a principal investigator, will be an intangible one: bringing together “a community of researchers who are dedicated to advancing research in a particular way.”
The grant for the SREE comes at a time when the Department of Education is experiencing major cutbacks. According to the federal budget released on Feb. 6, President Bush is cutting discretionary budget for the department by over $1 billion for fiscal year 2007, which is on the heels of more than $624 million in cuts for 2006.
Given that money is so tight, it may seem surprising that the department would dedicate $750,000 to founding a new research society that is expected to have no more than 3,000 members.
But according to Constas, who spent three years working for the U.S. Department of Education before coming to Cornell in 2003, his project is directly in keeping with the department’s strategic goals – a set of guidelines that it must report on annually.
One of the department’s strategic goals is to have 75 percent of its research budget go to supporting experimental investigations, which is the type of research that the SREE will encourage.
Experimental investigations often depend on randomized field trials (RFTs), which randomly assign students to control or experimental groups to evaluate certain education initiatives. RFTs have long been considered “the gold standard” in education research because they allow researchers to study isolated variables in a controlled setting.
Such clinical trials are common in medical research, and many argue that people should use similar methods to test education initiatives. That way, researchers can determine what curriculum or learning environment yields the best results, just as they would judge the effects of a particular medicine or medical procedure.
But RFTs are much less common in education research than more descriptive, anthropological methods.
“Educational research has become dominated by forms of inquiry that are as far as possible from RFTs,” said Prof. Stephen Morgan, sociology, who has written on research methods in the social sciences as well as on education. Morgan says education researchers mostly engage in “descriptive and ideological” studies that often give normative prescriptions without empirical findings to support their claims.
Constas and those joining him in this project, including co-founder Larry V. Hedges, a prominent education scholar at Northwestern University, hope to bring experimental design to the forefront of education research – a plan that has garnered mixed responses from the education community.
This is because the creation of SREE taps into longstanding controversy among education researchers regarding the use of RFTs.
Morgan praised RFTs in ideal conditions: “Randomization of treatments – assigning people to alternative policy interventions – is the most powerful way to evaluate alternative policies when you have control over the subjects and can closely monitor alternative treatments,” he said.
But in the real world, conditions are not always ideal. “Sometimes you can’t maintain the control, so people evade the treatment,” he said.
Critics of RFTs point to ethical and logistical concerns with assigning children to different treatment groups. After all, how many parents will sign on to let some scientists tinker with their children’s schooling?
Some researchers worry that RFTs will detract from other, less quantitative forms of education research – research that they say can better capture the complex environments in which learning does or does not take place.
Unlike observational studies, where researchers analyze the effectiveness of different programs already in existence, RFTs might end up investigating experimental approaches that could not easily be implemented in the real world, critics say.
And of course, money is an issue. Effectively conducting a randomized experiment depends on large-scale interventions, a full staff of paid researchers, and sometimes economic incentives to ensure the cooperation of schools and parents. After all of the effort, researchers end up with very good data on a very specific policy initiative.
In contrast, large-scale observational studies or surveys may be as expensive, but provide datasets that can be used to analyze many different policy initiatives over many years.
Constas hopes that SREE will allow researchers who use RFTs to come together and discuss how they deal with problems of logistics, ethics, and funding. But he emphasizes that the society is not restricted to those who use that particular methodology.
Nonetheless, according to a February 1 article in Education Week, many education scholars are worried that this new organization will further divide the community of education researchers based on their methodological preferences.
As Ellen Condliffe Lagemann, former dean of Harvard University’s school of education, told Education Week: “I believe generally that we need organizations and forces that pull everybody working on education research together, and that don’t fragment our efforts.”
But Constas rebuffed claims that SREE will divide the community. He plans to remain a member of established national professional groups such as the American Educational Research Association (AERA), which has 22,000 members, and says the others in his society will do the same.
In his own work, Constas has performed quantitative modeling studies that use regression analysis and do not involve manipulation of groups. He also has done qualitative work. He says the last time he did an experiment was “as an undergrad doing studies with pigeons in a psych lab.”
If anything, Constas thinks the existence of SREE will help its members feel more comfortable in the larger research organizations. He says the researchers can be happier in the umbrella groups, like AERA, if they “are not dissatisfied and don’t feel intellectually homeless.”
Archived article by Samantha Henig
Sun Staff Writer