A Simple Idea to Reform Standardized Testing

8

July 27, 2013 by juicefong

Ugh. Standardized testing. Such a double-edged sword. On the one hand, it gives us a faulty yet still meaningful measurement whereby we can assess school’s performance on teaching and learning. On the other hand, tests have been overemphasized by schools because they are so high-stakes. On the low end, teachers are doing test prep year-round. On the high end, great public schools feel handcuffed and have to dumb down their curriculum to optimize against tests that are actually a low bar for learning.

I’m not ready to remove standardized testing altogether, though. Even if you have a great school in middle class suburbs, you still want to make sure various subgroups are being served equally—across low-income students, English Language learners and all racial groups. I considered just doing random sampling of students within a school (say 25%), but you may not have enough of a sample size to measure these subgroups. You could test every other year or every three years instead of each year, grades 3-8 per No Child Left Behind. But then I worry that there may be too much slide year-to-year and the urgency would evaporate.

In talking to Dr. John Thompson yesterday (whose beliefs will appear in my upcoming “What’s Your Beef” series), a thought came to me: What if we tested students in the beginning of the school year, rather than the end? I could see this accomplishing a few things: 1) The test could be used much more like a diagnostic, and if teachers received an item-by-item analysis in the first month of school, that would help them decide how to spend the rest of the year with their students. 2) Test results could still be used as one way to assess the quality of schools. We’ll get to how this would work in a second. 3) By testing at the beginning of the year, schools would probably cut down on test prep, and might even do more to prevent summer learning loss. 4) This would blur the lines that directly tie teacher performance to high stakes test scores, in a productive way.

Tests as diagnostics, finally
The diagnostic element should be pretty clear. If students take a test in the first week of school, let’s revise the system so that test scores can be made available just a few weeks later. That’s both multiple choice and open-ended responses. Then let’s make sure teachers have a great analysis of strengths and weaknesses of all of their students by various standards or sets of skills, down to the specific question, to inform their teaching for the rest of the year. When I was a teacher in New York, we gave the tests in March and got the scores around May, but all we got was the scale score for each kid. We didn’t get a question-by-question analysis of how kids performed. Arm teachers with that kind of data in the fall and the test can truly be used as a diagnostic tool.

One new thought I’m adding in: because the sixth grade teacher gets her results that are mostly dependent on what happened in fifth grade and before, it might naturally lead to her having conversations with the fifth grade teachers and others. “I’m looking at the kids’ data, can you tell me what your curriculum was like last year? What do you think worked or didn’t work? What worked with this student?” When we test at the end of the year, the data for a cohort seems to rest squarely on one teacher, proliferating that “teacher as island” trap. Under this new model, it would encourage great collaboration between colleagues about their teaching and learning.

Tests can still play a part in measuring school performance
This data can still be used to softly assess school performance. We’d still test every child and I’d even be willing to do it every year, 3-8 per NCLB. We would still see trends in performance by various subgroups and we’d have a good picture overall at where all students stand as entering fifth graders or eighth graders. Of course you have transition years (e.g. a school that begins at sixth grade owes its sixth grade scores mostly to students’ previous school), so this gets a little tricky, but there will still be plenty of data to give you one snapshot of the school’s performance.

Dr. Thompson described to me how test scores in England are sent to an inspectorate, who every few years comes to the school and does a full assessment; presumably observing classrooms, interviewing teachers, students, parents and looking at achievement and other sets of data, such as teacher turnover or student attendance and suspensions. Gosh, that seems sensible. When I worked at a charter school in NYC, our authorizer did exactly this. A group of four people camped out with us for the better part of a week, observing classrooms and talking with various constituents and looking at data. The report they produced was full of helpful guidance—it wasn’t “gotcha” accountability. All schools deserve something like this: I could see standardized test scores contributing to that assessment and I could see value in doing an inspection every few years, as school quality does not typically change that much year-over-year.

Cut down on test prep and summer learning loss
This is a rather simple argument, but testing in the first or second week of school would hopefully cut down on test prep. Test prep seems to be most intense in schools in the months and weeks leading up to the high-stakes exams. Hopefully we’d see schools get away from that and instead offer great instruction year-round, knowing that good teaching, not test prep, is the best cause of students performing well on standardized tests. Currently, I see some of the best instruction in schools after testing period is over and the pressure is off. How ironic, right? By testing in the beginning of the school year, maybe we’d also see schools do more to prevent summer learning loss, and maybe summer school programs would actually be rigorous and helpful—most I know of in big urban districts seem to be lackluster.

Blur the straight line between teachers and test scores
Finally, and perhaps to the cheer of many teachers: by testing in the fall, we could blur the direct line between teachers and test scores and once and for all soften the notion that good teaching only means students doing well on standardized tests. If you’re a principal or anyone evaluating teachers and you’re relying solely on the results of one test to assess your teachers, then you’re just terribly mistaken. Teaching is more than what shows up on standardized tests. It’s about caring for and nurturing your kids, it’s about expanding their minds, it’s about helping them see a world they don’t know, it’s about inspiring them to greatness, it’s about filling them up with knowledge and ideas.

Don’t get me wrong, a good teacher should also see students do well on tests, but that is a low bar, and it’s just one bar. The trap we’ve found ourselves in with standardized testing is that it’s the only bar all of a sudden, because of how high-stakes they are and in the absence of less concrete and quantifiable ways to assess teachers. It’s time for districts and principals to put some real thought into how they evaluate their teachers more holistically.

Now, if I’m a fifth grade teacher, you better believe I want to know how every one of my students tested in the fall after they’ve left my classroom, and this data should be made available on the school level. But by just slightly blurring the connection between teacher performance and standardized test scores, we start to open the playing field to consider a variety of ways in which we assess teachers and perhaps we more strongly consider the notion that it is a holistic set of factors that influence student outcomes.

This is a new thought from the past 24 hours, so please send in your comments and poke holes in my argument.

Justin Fong (@jgfong) heads the small but mighty internal communications team at Teach For America. He writes freely about whatever he feels like, and of course does not expect his views to represent those of TFA or its leadership, ya know?

8 thoughts on “A Simple Idea to Reform Standardized Testing

  1. Hannah says:

    I like the idea of making tests purely diagnostic. This is something Sir Ken Robinson, an expert on learning and creativity, has always suggested. I agree with all the points that you brought up, especially the point you made about how it will lower the high stakes currently put on testing and the high pressure put on teachers to teach to the test. But what are your thoughts on the tests’ effects on students still? Especially if this is how they are welcomed back to school at the start of the year?

    Also, what kind of tests did you have in mind for the diagnostic exams? I would like to suggest that the tests be assessments of deeper learning that assess skill and performance with content rather than narrow assessments of multiple-choice that don’t fully encompass student ability and instead reduce student learning to meaningless metrics. Tests can provide useful data, so shouldn’t we give students the best tests that will provide the most useful data?

    If it were up to me there would be none of the standardized tests that we currently have. Tons of research show the negative effects of the current testing model on students and schools (and I’ve seen the effects with my own eyes, even in my own family) and I think that we can do better by our students if we move further away from traditional multiple choice testing towards methods of assessment that are also productive, in that they encourage student learning simultaneously.

    My current vision for “accountability” and “assessment” in school is portfolio and project-based, as standardized tests that are merely multiple choice tend to omit a wide range of skills and personal assets that are important to ensure in an education. I actually don’t see portfolios much as assessments, but more like educational tools, as students are able to learn, exercise their creativity, and demonstrate the interconnected mastery of essential skills (reasoning, research, argument, craft, etc…) as they are being evaluated for their progress. Portfolios can also give a more vivid picture of student progress while test scores show more outcomes. They are truly assessments of the deepest learning (see writings of Dr. Linda Darling-Hammond) that encompass student learning in a well-rounded and accurate manner. I think we can get a better picture of a student’s mastery of a skill through the work that student produces rather than his/her answer on question #46 of the state standardized test, right?

    What’s even better: these assessments will not only capture student performance in a more holistic manner, but will also allow for teachers to foster student growth through many unrestricted pathways. What I mean by this is that these pathways can involve projects that incorporate skills and concepts across various academic disciplines, especially in higher grade levels, accomplishing a lot more with a lot less. On the other hand, test questions test skills individually and on a shallow level; they don’t allow much room for the application and connection of different skills like student-produced projects do. For instance, a project for a unit on the Civil War can include a research paper on an event/aspect of the war (this helps students develop research/writing/argumentation skills) paired with a creative piece (video, 3D design, poster, diagram, song, etc…) that perhaps connects the Civil War to 21st century society (this helps students sharpen the skills of application, connection, creativity, and critical higher-order thinking). The possibilities are endless! I think there is a unique opportunity here for students to get a layered, interdisciplinary, and multi-faceted education.

    Now I’m going to go into the logistics of all this. These are ideas that I’m still developing as well, however, so feel free to comment on or question anything you see!

    I’m envisioning that portfolios would consist of the highlights of a student’s work throughout the course of a school year and would be submitted for review at the end of the year. Teachers can be trusted to assess student work in written evaluations and a third-party can intervene every year or every few years to go over student work in schools (as well as observe classrooms like you mentioned… I really like that idea; it was something my own school had and it worked very well). This might take more effort (and perhaps time, although I do think it’s time that’s better-utilized than test-prep or test-scoring), but students are worth the investment and if it happens on a local level, it won’t be too tough.

    Standards that portfolios will have to meet will build upon each other rather than be unique to a sole grade level (so perhaps we can improve upon Common Core, this time with more student/teacher/parent input). These standards can serve as sort of a flexible rubric for student work portfolios (flexible in that students can demonstrate mastery of a skill of concept in many different ways, but all will have mastered the skill).

    As for the diagnostic part of this, teachers would be able to receive their students’ portfolios from the previous year during the summer so they can be able to look at their incoming class’s abilities. They can also do a diagnostic assignment at the beginning of the year, something perhaps a little more exciting and engaging than a test, and use that data to figure out the best paths to take for their class. (side note on this: I think it’s really important during the first week of school for teachers to not only lay down expectations but also listen to their students intently and figure out what they expect.)

    Let me know what you think of this idea, and if I missed anything important! Always eager to hear different perspectives. Thank you for your insightful post!

    • juicefong says:

      I don’t think my proposal excludes your idea of portfolios, which I believe used to exist in many districts but have been wiped out by standardized testing. I agree that portfolios can really show what students are capable of and can demonstrate deeper learning, but they are also not very standardized, which becomes problematic. Given that, I think standardized testing can and should still play a role because they offer more of an apples-to-apples comparison.

      While it would be great for tests to look at deeper learning, notice I acknowledge that they measure a low bar of learning. I would leave portfolios and other ways of assessing individual progress to the task of more fully examining student growth. If we test in the fall, you probably still want to lean multiple choice, at least out of expediency. The more complicated it gets, the more open-ended questions to grade, the greater the lag in getting data together for teachers to work with. There’s room for both, but there’s also a trade-off.

      I’m reading directly from a New York State fifth grade math test: “Dante’s bath towel is 1 yard long. What measure is equivalent to 1 yard, in inches?” That’s not unreasonable to ask.

      • Hannah says:

        You make a good point about expediency. Diagnostic data does need to be made readily available so that teachers can know what areas they need to cover first to make sure all students are on the same page with the basics (like the sample question you offered). That’s a fair point. While some might say that that could lead to teaching to the test because then the teacher would need to devote time to test prep (just now at the beginning of the year) to get students up to speed, I don’t see why that has to happen. After the diagnostic exam, the project-based learning and portfolio-based assessment could kick in and first target the standards that were not met in the exam.

        But then again, couldn’t a student’s portfolio from the previous year serve as a factor in diagnostics? And do you think it’d be good if the teacher still had a diagnostic project of sorts that was designed to look at where students are at with other skills that a test cannot measure? I’m just kind of thinking out loud here; would love to hear your thoughts too.

        Also, I’m not sure if you meant it this way when you said “apples-to-apples” (love that game btw… anyway…), but I just want to stress that I think it’s really important for us not to use scores to compare students. We should focus on fostering the growth of the individual. The effect of comparison and competition on students and an entire school for that matter has been really devastating to witness… and it’s something I hope we can move away from. I think your idea of using tests at diagnostics moves us closer to a more collaborative and encouraging learning environment centered around individual growth.

    • Mark Hurty says:

      Hannah and Justin: great conversation.

      The biggest problem with standardized testing is that it creates a false perception that children who score similarly on the test have been given an equal education. If education were simply about diagnosing and correcting deficiencies in knowledge or skills, we’d be fine with the system we have now under NCLB. But education — the opening of the mind — is about creating a fertile environment where a student is nurtured and encouraged, where curiosity is fostered, and differences are cherished and honored. Some children are great writers, singers, actors, and painters. Others are mathematicians, scientists, philosophers, and logicians. Of course the goal of an educational system should be to foster all kinds of students.

      Our current system of standardized testing can’t help us with this. Switching to a diagnostic rather than summative system of standardized testing may make those tests more useful for diagnosing the very narrow scope of what they asses, but they’re still woefully inadequate for assuring that no child is left behind. (An aside: the law should have been called “no child gets ahead.” As with most of the Bush era legislative initiatives it was named ironically.)

      Hannah’s idea of portfolio based assessment is a better model, but it’ll be difficult to convince legislators, governors, departments of education, and their ilk. To the political class, opposing standardized tests is like being soft on crime. It doesn’t matter that the tests aren’t able to bring equity to our system of educating our children. They are a dumbed down proxy for education, proving only that kids can be trained to perform well on tests.

      And of course there’s the issue of scalability. Teachers at the middle school where I taught were responsible for a single subject and about 130 kids. That’s a big number of portfolios to asses each year. Not that there aren’t creative ways to manage this volume, But it might require a change to our current classroom schedules.

      One of the things that Ken Robinson noted in a recent talk he gave —
      http://www.youtube.com/watch?v=9HRXhRjfS7Q — was that more of the responsibility for deciding what and how to teach should rest in the hands of educators, starting with teachers. That won’t happen until we start to undo some of the damage done by the worst parts of NCLB (NCGA).

      In his talk he elaborates on “theatre” as a metaphor for education first suggested by British director, Peter Brooks. He notes that Brooks suggests the best theatre is a production that strips away and simplifies until you’re left with just the essential. Just actor and audience. Only those elements that enhance the experience are permitted on stage.

      To apply that metaphor to our education system: we should revere the relationship between teacher and student, bringing only those essentials into the classroom that enhance the mutual journey that students and teachers join towards the goal of ever more creative, critical, and open minds. Instead of subjecting children to rote learning, preoccupied with picking the single right answer from a group of four choices, we should be thinking about making K-12 school more like an institution of liberal arts. This would begin to address educational inequity in this country, by focusing on the things that help children become curious, lifelong learners, possessors of open minds, able to think critically and creatively. Show me a standardized test for that.

  2. […] While I do have my own… often critical opinions of his other pieces about TFA, he published something today that I found very insightful (and you should definitely check it out!) I took the time to write a reply, offering further thoughts on how we can change standardized […]

  3. […] A proposal to administer standardized tests not at the end, but the beginning of the school year. (Fong) […]

  4. […] Fong proposed on his blog Fongalong recently that state testing should be moved from the spring to fall to lessen the reliance on […]

  5. Alec says:

    First, summer learning “loss” is somewhat of myth. Students of privilege, of which I include most middle class, suburban kids, have numerous enrichment opportunities from museums to summer camps to simply reading. They actually gain in knowledge and critical reasoning skills. Meanwhile, those on the other end of the opportunity gap don’t necessarily lose knowledge, but the gap widens as they stagnate over the summer. It is an opportunity gap, not an achievement gap.

    Secondly, how do you deal with the incredible mobility of urban schools?

    Thirdly, do you honestly believe that all the good teachers just randomly, by some cosmic quirk of fate happened to land in suburban New York, and all the lousy teachers just happened to land in the city? That is what standardized tests tell us. That is all they will tell us if we use them that way.

    Fourth, good teachers have always used formative assessments in concert with their colleagues to guide instruction, fill gaps, seek out help.

    Fifth, I can have the same course, taught different hours. Because placement of students is rarely random, once section can make a teacher look like a superstar and the other section make them look like a goat. The reality can even be that the teacher could do a better job and work harder on the class whose test scores are worse. Some years you might have all the challenging kids, some years easier kids. Standardized test scores tell very little. Unless of course you believe in the lousy teacher narrative.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: