Response from Erica Lembke
I think this is a really good question because too often teachers are collecting progress monitoring data, but they are not really utilizing the data and we know that that is really the key to seeing change, positive change, in your student's performance is really utilizing that data. So, it's an excellent question and teachers should be looking at and attending to the data all the time. It's interesting, sort of an interesting tangent is, who is the person that should be attending to and looking at this data and thinking about the change that should be made? I get asked this, should it be the general ed classroom teacher, should it be the interventionist, should it be the special education teacher and so it's important to think about in your school, in your district, who the person is that provides that primary reading instruction to the student. Sometimes that's the gen ed teacher, sometimes that's the special education teacher and that's the person I would recommend collect that progress monitoring data and think about making those changes in instruction because that's the person that is primarily responsible then for that instruction, ultimately.
We have to go back and remember that progress monitoring data; that data is not just to tell you, oh students good or bad, that data is to inform teachers instruction. That's really the key behind that. So, after you've collected some progress monitoring data, it's important to collect enough data so that you let the intervention play out long enough and when we think about the differences in how quickly we make a change when we have behavioral data, graphed behavioral data, or graph academic data, it's not all synchronous. Behavior data, you might have a student that has challenging behavior and we may go in and make a change much more quickly based on that data than we do with an academic intervention. So the general recommendation or guideline from the field is that we need at least, most researchers would say, at least eight data points. Some would say as many as ten to twelve data points before we go in and look at the data and make a change in instruction. And I want to back up and say of course you are looking at the data each time you graph and enter that data however in looking at it to really think about whether an intervention change is needed.
The reason we suggest collecting so many data points, because some of you may say wow that's a long time to wait for a particular student, but the reason we suggest waiting a while to make that change is because we want a really stable indicator, or stable slope of how the student is performing. Eight data points collected across at least four to six weeks gives you a nice, stable trend of student performance. That really brings me to how do I make that decision; there's a couple of ways that you can utilize to make an instructional decision.
One would be, and these are just some basic guidelines, one might be the four point rule where you actually look back at the four most recent data points that were collected, again understanding that we are talking about three to four weeks of data, eight or so data points, look back at the most recent four data points. If all four data points are below the goal line make a change in instruction. All four points are above the goal line raise the goal. That's a great situation; the child is doing really well. Data points both above and below the goal line continue with your current instruction. And it's important to pick a decision-making rule like this and stick with it because you do see graph data sometimes where a child, I saw it recently, I saw a graph where a child had three or so, almost four data points sort of heading downwards and then kind of was back up on the upswing. However, this was a child, a seventh-grade girl, reading in the third-grade level, goals set very low, you know, and so if we didn't have a decision-making goal the tendency might be to eyeball that data, and say "oh she's goal, she's back up on the upswing" you know "I think we'll wait a few data points longer," when in actuality when we looked back at that data, her data points were indicating her performance had been low for a while. So it's important as teams, and as teacher teams, that we have those clear, sort of concise decision making rules, and that we stick with them.
Another example of a decision making rule that is more accurate, maybe in some cases a little bit more difficult to calculate, would be the Trend Line Rule. We look back essentially at the eight data points or so that we collected so far, we map a trend line or a line of best fit onto that data. Some of your computerized CBM programs now actually put that trend line in for you. You look back, and you compare once again, just like we did in the four point rule, to the goal line. So you are comparing trend of student performance with goal that was set for the child. If the trend of student performance is less steep than the goal line it's time to make a change, an instructional change. If the trend of student performance is greater than the goal line, great situation again, student is doing better than we thought, maybe raise the goal line. If the trend is about parallel and close to the goal line we can continue with our current instruction. The reason that the trend line, that method of data utilization is more accurate is that we are actually using more of the data to make our decision. We are not just using the previous four points; we are using a lot of the data to make that decision. I've actually put those types of rules, those choices I guess for decision making rules and kind of put those into a rubric that helps teacher look at academic performance, behavior in the classroom, data from other assessments, to make decisions about movement between tiers. Because, again, we don't want that to be an arbitrary decision either.
We want to systematize this process and so we think about any time, and this is a really critical point, we think about, not just, oh okay I'm going to go in and make a change now but before we ever make a change we want to think about has the instruction been delivered with fidelity, have I been as intense as I've needed to be as about instruction. Perhaps I wanted to deliver intervention for thirty minutes, you know, four or five times a week and intervention time is right next to recess and then I only get to do intervention for 15 minutes you know each day. Intensity, fidelity, specificity of instruction, do the teachers understand the specifics of how that lesson should be taught? So it sort of goes back to effective teaching behaviors so before we ever make that change then we want to be careful to look at, and address some of those questions as we utilize that progress monitoring data.