Teaching kids to read and helping those who struggle

Reading Rockets offers a wealth of reading strategies, lessons, and activities designed to help young children learn how to read and read better. Our reading resources assist parents, teachers, and other educators in helping struggling readers build fluency, vocabulary, and comprehension skills.

Home > Reading Topics A-Z > What "Research Says" Really Means

• 
• 
• 

What "Research Says" Really Means

The statement "research says" can be used to mean a variety of things. Find out a classification system for the reliability of research findings so that you can evaluate the strength of research-based recommendations for instruction.

Ellis and Fouts (1993, 1994) provide a classification system that is helpful in evaluating the strength of the evidence behind the statement, "The research says…" Their 3-level classification system is somewhat revised below:

Levels of research

How much evidence is there to support a theory?

Level 1 – "Basic" research.

Are there correlations? Is there a rational explanation (theory) for these correlations?

Level 2 – Test of the theory in real classrooms.

In small-scale comparative studies, does the theory accurately predict which practices will result in better learning?

Level 3– Program evaluation on a school- or district-wide basis.

In large-scale comparative studies, does the theory predict?

Shared knowledge base

• Information taught to preservice teachers
• Information widely disseminated to the field of inservice teachers

According to Ellis and Fout's classification system, there are three levels of research. Level 1 research is "basic research on learning." Correlations, descriptive data and qualitative case studies comprise Level 1 research. Level 1 research is abundant. However, no theory regarding teaching procedures is testable with descriptive and correlation research (Level 1) no matter how abundant it is.

For example, from the correlation between high achievement and high levels of self-esteem, some have concluded that self-esteem causes achievement and that by offering warmth and sympathy to children who fail will build self-esteem and higher achievement will result. This is not necessarily true. Take for example another correlation that shows that people with higher achievement have larger shoe sizes. The correlation is explained by a third variable: as children grow older they achieve more and their feet grow.

Correlation data can, however, be used to disprove a theory. If there is no correlation between two variables, one variable certainly does not predict the occurrence of the other.

At Level 2 a theory describing how teachers should teach is tested by applying it in the classroom to see if it accurately predicts and gets better results than the practice it replaces. Different teaching interventions are compared at Level 2 in controlled research studies to see if students learn more or better in classrooms using teaching procedures based on the theory. Students are randomly assigned to two or more instructional groups. One group learns one way, the other group(s) learn other ways. The results are compared. Researchers use statistics to decide whether any differences in the results were accidental or would be likely to occur again.

Level 2 tests of the hypothesis that expressing warmth and sympathy toward students who fail actually leads to a deterioration in self-esteem. For example, Graham (1984) found that warmth and sympathy towards students when they fail to succeed on school tasks serves to reduce further the students' beliefs in their own capabilities. According to Ellis and Fout's review of the research, mastery learning and cooperative learning are two practices that have strong Level 2 support.

Level 3 research evaluates the effects of the recommended teaching intervention in school-wide or district-wide implementations. Level 3 is important because at this Level the new intervention is integrated into all the other things that teachers must accomplish in a day.

The danger of having only Level 2 research support is that we may find that something is very good for reading, but when we get to Level 3, we see that it takes so much time that it interferes with teaching other subjects such as math.

For example, cooperative learning may get good results in reading at Level 2, but may require so much time to get these results, that fewer learning goals are accomplished. This shortcoming would become evident through Level 3 research. At Level 3 scientists are not evaluating one hypothesis regarding one tool, but are evaluating the integration of a whole toolbox full of tools to maximize effectiveness.

Education is different from the field of science in that education reform leaders tend to call their hypotheses "theories." In science, the word "theory" is used to describe a hypothesis that has been tested.

We can compare Ellis and Fout's three-level analysis with the traditional scientific method. Level 1, theory building in education, is equivalent to hypothesis formation in science. Level 2, theory testing, would be the actual experiment. In science, we wouldn't use the word "theory" until after Level 2. Level 3 goes one step beyond the basic scientific method but seems necessary, given the tendency in education to pick up one new tool, even one that works, and carry it to an extreme, throwing out all the old tools though they are still necessary to do the complex job of teaching.

Constructing knowledge from evidence

Education	Scientific Method
Level 1 – Theory building	1. Develop a hypothesis through informal observation.
Level 2 – Test the theory	2. Test the hypothesis by formally attempting to disprove it. 3. Analyze the data from the test to evaluate the truth of the hypothesis.
Level 3 – develop high performing model schools

Many reformers emphasize the importance of "new" research and do not acknowledge any contribution from "old" research. If medicine functioned as education does, we would see large numbers of people suffering the debilitating effects of polio again, because the Salk vaccine, a discovery of the 1950's, was "old" research.

An educational system that looked to Level 3 research would be more likely to integrate old as well as new research in a deepening understanding of the relationship between teaching and learning. To use only the most recent research to build our professional-knowledge unnecessarily limits our understanding. The nature of children's learning has probably not changed much in hundreds of years.

Research is timeless. We can look back at old studies, look at the instruction, the measures, and the results and integrate these results with new research. In this way our professional-knowledge base grows over time and becomes more refined.

The professional-knowledge base for the teaching profession should come from high-performing schools. The ultimate measure of predictability is a Level 3 demonstration of a high-performing model that is replicable. A high-performing model that is not replicable is not as useful to the profession as one that is replicable. A replicable model can be used to teach the rest of the profession how to get high achievement levels. A high performing model would be replicable if more than one high performing school can be developed from the same model.

Because community wealth is such a strong predictor of performance Level, only schools with similar socio-economic levels should be compared to identify high-performing schools. This way the effects of the instructional practices, rather than the wealth of the parents, can be seen. It is also possible that different intervention models will work better to solve the different problems that distinguish low income schools from high income schools. We would learn if this is true or not by comparing schools only with other schools of similar socio-economic status.

If high-performing schools (Level 3 research) were the gatekeepers for new knowledge in education, then these working models of high performing teachers would become the dissemination centers for educational reform. The practitioners and researchers who actually get the results this country is demanding would become the leaders of the educational reform movement and teach other practitioners across the country how they, too, can achieve at high performance levels.

These high performing schools could serve as training sites for preservice teachers as well. In a system that looked to high-performing schools, rather than to the gods, theories that have not yet produced results could not be widely disseminated. Faddism would not be able to get a foothold.

This is not to say that theories that are only at Level 1 have no merit and will never work. It only means that new teachers should not be trained in theories with only Level 1 support and that districts should not mandate practices or spend large amounts of money promoting teaching practices with only Level 1 support. These restrictions would not prevent individual teachers from reading about Level 1 research and working with it to see if promising interventions can be developed from it. Anyone using Level 1 research would understand its limitations as such. It would not prevent school communities of teachers and parents from working with their favorite researchers and theorists with a goal to develop a new model of a replicable high performing school. But Level 2 research would precede this kind of initiative, to reduce the educational risks to the larger number of children involved.

Level 2 research would remain very important for high-performing schools because Level 2 comparative studies would allow identification of the specific components that are important for replicating the high performance. Only the teaching procedures that gather Level 2 and 3 support should become part of our teacher-training programs and become widely disseminated across the profession.

Uses and abuses of research

A huge problem in education is that most innovations jump from Level 1 straight into the "shared professional-knowledge base" without being tested. Most of the educational practices that become widely disseminated in our university teacher-training programs and across the nation do not even have Level 2 research support, nevermind Level 3.

Other professions have well-established gatekeepers that monitor the information that enters the shared professional-knowledge base. For example, the Food and Drug Administration is a gatekeeper for the medical profession.

In education though, the trend is to test each new hypothesis on a nationwide scale. When the result is a national failure, who gets blamed? Not the promoters, not the consultants providing staff development, not the university professors. The teachers are blamed for the failure: "The teachers didn't do it right."

To ask for Level 2 research is to ask the people who are telling teachers how to teach simply to say, "Show us how it's done. Once you show us how to get these children performing at noticeably higher levels of performance then we'll take a look at what you've got to say. Don't bother us before that."

References