Maybe it’s because of a certain cognitive bias in me owing to heightened awareness, but for whatever reason, I have been finding a lot of stuff about data collection lately and associated issues, including a piece on a paradox inherent to assessment. According to author Victor Borden:
Information gleaned from assessment for improvement does not aggregate well for public communication, and information gleaned from assessment for accountability does not disaggregate well to inform program-level evaluation.
But there is more than just a mismatch in perspective. Nancy Shulock describes an “accountability culture gap” between policy makers, who desire relatively simple, comparable, unambiguous information that provides clear evidence as to whether basic goals are achieved, and members of the academy, who find such bottom line approaches threatening, inappropriate, and demeaning of deeply held values.
For anyone who’s been involved with or aware of assessment at HW, you’ll know that Cecilia came in and consistently coached toward the idea that assessment ought to be formative–aimed at building knowledge about student learning in order to make changes to practices whose impact could, theoretically, at least, be measured.
Of course, assessment is (or can be) used for other purposes, too–an assessment that tells us that more than half of our students who have completed their Gen Ed Humanities requirement cannot (or will not) write a series of short essays on an art object that meet our expectations for what students should be able to do provides useful information (internally) that is potentially destructive and harmful to the institution and its members (even the students). So there is, in every assessment, a tension at least and a temptation to resolve the difficulty by cooking the measure (or the numbers) to make sure we look good. Having worked on the Assessment Committee for six years I can attest that we work hard to avoid those traps, but we’ve certainly had discussions about the potential impact of certain findings.
Anyway, this article does the best job I’ve ever come across of explaining a dynamic that I’ve long felt, but never been able to pin down. Well worth the time to read it, I’d say.