I have used a lot of ink over the past three years extolling the virtues of our teacher leadership system and sharing with you all the good work that we have been able to do under this system. By distributing and flattening our leadership structure, those closest to instruction have been given the authority to make decisions about what works in classrooms. Operating within a defined set of parameters that have been outlined by the Board of Directors, we are able to stay true to our district priorities that include a focus and emphasis on literacy, mathematics, and technology integration.
From a data collection standpoint, it has been (and will continue to be) difficult to draw a straight line from the work of our teacher leaders to performance on the statewide standardized tests known as the Iowa Assessments. I have carried on relentlessly about the fact that the Iowa Assessment does little to measure the effectiveness of instruction in the classroom. I say this because we know that the Iowa Assessment doesn't align well with the Iowa Core Academic Standards. And unfortunately both the Iowa Assessment and Iowa Core Academic Standards are legal requirements that, frankly are the antithesis of one another. So instead, to measure the effectiveness of our teacher leadership system we have relied on a lot of qualitative measures.
From a data collection standpoint, it has been (and will continue to be) difficult to draw a straight line from the work of our teacher leaders to performance on the statewide standardized tests known as the Iowa Assessments. I have carried on relentlessly about the fact that the Iowa Assessment does little to measure the effectiveness of instruction in the classroom. I say this because we know that the Iowa Assessment doesn't align well with the Iowa Core Academic Standards. And unfortunately both the Iowa Assessment and Iowa Core Academic Standards are legal requirements that, frankly are the antithesis of one another. So instead, to measure the effectiveness of our teacher leadership system we have relied on a lot of qualitative measures.
The qualitative measures we collect suggest to us that instruction is being strengthened in our schools and we have in fact tightened alignment to the Iowa Core Academic Standards. We have done this by ensuring foremost that what is being taught is what is being assessed. Beyond that, once we administer our assessment; those students who have not yet mastered the content receive supplemental instruction. Now, both the instructional strategies that are being used to deliver the core content and the supplemental strategies that are being deployed for remediation have been researched, developed, designed, field tested, and ultimately delivered by our teacher leadership team. The million dollar question(s) then become, do they work; and how do we know? The short answer is yes, and this where we have hard quantitative data to back up our claim. I am going to get a bit technical, but please bear with me because this is not only fascinating, it is AWESOME!
For this we turn to a statistical measure known as effect size. Effect size compares the difference in scores in a pre-test and a post test analysis. Obviously post-test scores are going to be higher just by virtue of regular classroom instruction. But what we are trying to determine with effect size is quite simply, to what degree is the measure due to normal growth or the introduction of a new strategy or technique? And more importantly, is the growth normal, or statistically significant?
As stated above, one of our district priorities is math instruction and as such we have invested a lot of teacher leadership resources into ensuring that our instructors have access to high quality math instructional strategies that are tested and researched as part of our professional development plan. What we are able to do with effect size then, is measure the success of these instructional strategies. By evaluating our math classroom assessment data in the fall and spring (2015-2016) for K-2 on basic number knowledge and on fraction assessments in grades 3-6, we have been able to measure the effect size. Basically, answering the question, does it work or not?
But before we get to that, let's take a look at what constitutes 'working'. According to the educational researcher John Hattie, an effect size of 0.40 over the span of one entire academic year could be attributed to the normal effect of a classroom teacher. So by virtue of regular classroom instruction we should expect to see an effect size above 0.40. Everything we do is going to have some sore of effect, right? But what we are trying to measure here is the degree to what effect. In the chart to the right, an effect size of 0.20 is considered small. Once the effect size surpasses 0.40, it is considered statistically significant, and that we can begin to notice 'real world differences' and can claim the innovation is effective.
Here is where it gets pretty interesting folks, because as the data table to the left suggests, the innovations are working. But not only are they working, they are working really well! Bear in mind we are looking specifically at the effectiveness of math instruction (number knowledge K-2, fractions 3-6). In every grade level cohort, this data seems to indicate that the effect of instruction is statistically significant. In other words, it goes well beyond merely showing up for class and doing the same things that we have done over and over again.
Now, for the sake of full disclosure I do have to offer some cautionary words of wisdom. For starters, our sample size is relatively small, which makes the data subject to more variation. Obviously we can't overcome that because the size of class is the size of the class. Additionally it is also important to note that this is only one data point and has not been calibrated against other data sources.
Nevertheless, it's pretty cool, isn't it? I would like to thank Joe Kramer from AEA 267 for helping with the statistics of this project and sharing these data points. He was incredibly helpful in the analysis of this data and answering my multitude of questions.
For this we turn to a statistical measure known as effect size. Effect size compares the difference in scores in a pre-test and a post test analysis. Obviously post-test scores are going to be higher just by virtue of regular classroom instruction. But what we are trying to determine with effect size is quite simply, to what degree is the measure due to normal growth or the introduction of a new strategy or technique? And more importantly, is the growth normal, or statistically significant?
As stated above, one of our district priorities is math instruction and as such we have invested a lot of teacher leadership resources into ensuring that our instructors have access to high quality math instructional strategies that are tested and researched as part of our professional development plan. What we are able to do with effect size then, is measure the success of these instructional strategies. By evaluating our math classroom assessment data in the fall and spring (2015-2016) for K-2 on basic number knowledge and on fraction assessments in grades 3-6, we have been able to measure the effect size. Basically, answering the question, does it work or not?
But before we get to that, let's take a look at what constitutes 'working'. According to the educational researcher John Hattie, an effect size of 0.40 over the span of one entire academic year could be attributed to the normal effect of a classroom teacher. So by virtue of regular classroom instruction we should expect to see an effect size above 0.40. Everything we do is going to have some sore of effect, right? But what we are trying to measure here is the degree to what effect. In the chart to the right, an effect size of 0.20 is considered small. Once the effect size surpasses 0.40, it is considered statistically significant, and that we can begin to notice 'real world differences' and can claim the innovation is effective.Now, for the sake of full disclosure I do have to offer some cautionary words of wisdom. For starters, our sample size is relatively small, which makes the data subject to more variation. Obviously we can't overcome that because the size of class is the size of the class. Additionally it is also important to note that this is only one data point and has not been calibrated against other data sources.
Nevertheless, it's pretty cool, isn't it? I would like to thank Joe Kramer from AEA 267 for helping with the statistics of this project and sharing these data points. He was incredibly helpful in the analysis of this data and answering my multitude of questions.
No comments:
Post a Comment