Posts Tagged ‘college board’

Movie Review: The Test and the Art of Thinking

Dear Reader,

Thanks ot a local homeschool group, I recently had the opportunity to watch “The Test and the Art of Thinking,” a movie on the SATs. I wrote this review for my local group and thought I would share it here as well.

Not surprisingly, this movie was critical of the SATs (and ACTs though less time was spent specifically on them) as criteria for college admissions. It began with a brief discussion of the original purpose of the test. This was not actually entirely bad. Though it was the era of eugenics and most scientists expected the test to show differences between those of us who are more evolved and those who are less so, it also had an egalitarian purpose. Prestigious schools of the day each had their own admission tests and only offered them to students who were already at high level prep schools. A common test allowed students from different backgrounds to compete.

The main criticism of the test was that it does not really measure intelligence. This is true for a number of reasons including: There is not just one kind of intelligence. It is very hard to measure or even define true intelligence. Beating the test has itself become a game of tricks in which those who can pay for expensive prep classes have an advantage.

There was also some talk of the power of the test in society. Though started by those not looking to make money, it is now big business. While some colleges have dropped testing requirements for admissions, the big names still use it and it is hard for others not to follow suit. It was implied that these elite colleges somehow must benefit from using the test though it was not specified how. The national rankings of colleges also play a role and people watch them closely and the average SAT score of admitted students is considered in them (though it was not abundantly clear to me how large this one measure plays in overall rankings).

This movie was best when it was specific and showed ways the test can be “gamed.” They demonstrated for instance that in the essay portion (which is no longer required or even wanted by most schools in my experience) that a long essay gets high marks even if its content is complete drivel. They also showed some tricks prep agencies teach for getting probable right answers without even reading the problem.

I had a number of issues with or questions about the movie:

  • It relied heavily on test prep people and admissions staff (or former admissions staff). Every time a College Board (the people who run the SATs) person was talking it was recorded from some other forum. It may be the College Board refused to talk directly to them, but then this should have been said.
  • It was very low on statistics. In fact, there were almost none that related to the success or bias of the test. There was an allusion made to gender differences but no facts on what these are. Again, it was said in passing that test scores do not correlate to college success and that all they correlate to is parents’ educational level (all things I have heard before in other contexts), but hard numbers to back these things up would have been more persuasive. Nor was there any real discussion of how poorer and otherwise marginalized groups do on the test.
  • There is no doubt schools rely on test scores. What I have heard is that even top schools do not rely solely on test scores. Harvard gets a lot of 1600s applying and they look beyond scores for something more. The movie presented things as black-and-white, we use scores or we don’t. I think an honest assessment would need to look at how schools really evaluate students and how much of a role those test scores actually play. (I know a lot of this information is proprietary and that schools do not want to share how they make decisions but we need to at least acknowledge that it is not a simple process.)
  • The movie is dated. Though it was made in 2018, the SAT has changed recently and the essay is no longer required and (from looking at schools for my son last year) most schools don’t even want it. The best criticism the movie had was of how essays don’t even have to be true (see above) but it is no longer relevant. I laughed in appreciation when they said the reading selections are like articles from Time Magazine and there are still a number like this, but my experience with my children is that they are also now including passages from real literature (like Jane Austen novels). In my observation there has been some real improvement in the latest changes which was not addressed.
  • Most of the tricks shown which cheat the system had to do with the math section. There may be similar tricks for the reading and writing potions but this was not made clear. So I am left wondering if those portions are also as game-able.
  • At one point one of the talking heads talks about a hypothetical question about who was president during WWII and how some answers, though wrong, are still better than others. I get his point, but it wasn’t well related to the test which does not have these sorts of factual history questions. I assume he was meaning to say something about the reading portion which often asks for the best answer out of a selection of possible ones but this connection was not made clear.
  • Obviously some people pay oodles of money to learn the tricks of the test. I would like to know how much they actually improve their scores by doing so. My kids who have taken the test improved some by doing practice tests at home. How does this method of preparation compare to those expensive classes? How much can a 1400 kid (on a first try) imporve versus a 1000 kid? Again hard numbers are needed.
  • There is an underlying value system here which I don’t buy into anyway which says that one needs to get into the elite colleges and therefore needs the best scores. When my own son was looking for colleges, we saw that the elite ones require a certain number of SAT subject tests or AP tests. Knowing he would hate to do all that extra prep and testing and feeling that it would be a waste of his time, we eliminated such schools from our list under the assumption that if they attract people that are so focused on such things they are probably not good schools for him anyway. It is hard to avoid the SAT (or ACT) in our society, but one can keep it in perspective and get by without buying into the whole system.
  • Not really a criticism of the movie: The test was not originally game-able (even in the 1980s when many of us parents were taking it, this was not a big thing). Since it has become so, because people have discovered ways to get right answers without actually doing the problems, the whole thing has become a game and of less value overall. The film used a lot of test prep people who make lots of money teaching rich kids how to trick the system. (I don’t honestly know how these people live with themselves, but that’s a side issue.) I think we should not be surprised that human beings cannot create an un-game-able test but how this comes through in test questions thereby making them game-able seems like it would be a fascinating psychological study to me.

“The Test and the Art of Thinking” did not really provide new information. I went in expecting it to tell me just the things it told me: that the test is game-able, that those who can afford expensive test prep have an advantage, that it does not measure true intelligence. I didn’t find that there was much new added to the discussion here and I would really have liked to see the hard numbers to back all this up.

Nebby

Sabbath Mood Homeschool

Desiring That a Sabbath Mood Rest on Your Homeschool

A Work in Progress Productions

Learn•Grow•Shine || Based in Attleboro, Ma

dayuntoday

my musings, wise or otherwise

Festival Fete

locally grown art, food, and merriment

StrongHaven

A Literary Homestead

journey-and-destination

Blogging about education, theology, and more

Harmony Fine Arts

Blogging about education, theology, and more

Sage Parnassus

Blogging about education, theology, and more

A peaceful day

Blogging about education, theology, and more

Living Charlotte Mason in California

Blogging about education, theology, and more

weeklywalrus

Weekly Walrus Whatevers

Creations by Maris

Handwoven Textiles

Fisher Academy International ~ Teaching Home

Blogging about education, theology, and more

Afterthoughts

Blogging about education, theology, and more

Leah's Bookshelf

Book Reviews You Can Trust

Duxbury Art Boosters

Supporting the visual arts in Duxbury Public Schools