How to Assess Students’ Abilities in Admissions Decisions

Over at his blog, my brother discusses the trend of schools not looking at the SAT scores when making admissions decisions (with his SHC at the cutting edge of that movement). It’s an interesting decision, and one that I find worthy of pursuit, but I also believe it is fraught with challenges. As I see it, there are few “objective” measures by which we can evaluate students. Of course there are many “quantitative” ways to assess students.

The first that comes to mind is their GPA. Unfortunately, this is, while certainly a “quantitative” measure, it is not “objective.” It fails in at least two ways. First, it fails to consider that variations in content covered (pace, depth, breadth, etc) and also cannot account for the subjectivity and variability of the grading process itself. At best it doesn’t allow for cross set comparison (comparing students from different schools), and at worst, it fails to accurately provide any useful information.

In a selection process, one needs to be able to compare, say, Chris to Drew. If I only have one slot left, how do I determine which of the two are more likely to succeed? Let’s say for the sake of argument that both have identical GPA’s, but are from two different schools. Are both schools the same? Did they cover the same material? Did the teachers evaluate the students in the same way, asking the same questions, and grading the exams and other work in a similar, and consistent, manner? This of course, gets more complicated if, say, one of you has a slightly higher GPA. Does that indicate anything in the cross-set comparison?

As I mentioned, at worst we get very little useful information about the student. Without an in-depth knowledge of the school from which they come, and ideally the specific instructors/teachers they had, we cannot infer much by their grades. Did one instructor grade harshly, while another applied mis-directed compassion? Perhaps there was pressure from an administration to increase “graduation rates” at their school, or conversely it is possible one school had a more restrictive grading scale. It’s fairly common for one HS to grade 80-89 as a B, while another grades a B as 86-93.

I am not arguing that the SATs (or GRE, or GMAT, or MSAT, etc.) are perfect. They are not. But they at least provide some common ground upon which we can evaluate a student’s knowledge (what they have been able to learn) and assess a student’s potential (their ability to learn.)

So–barring a nation-wide attempt to standardize instruction, or even more challenging, document the experience in every classroom in the nation, how does a school effectively make useful quantitative (i.e. measurable) and objective comparisons and assessments of ability and performance?

Anyone?