Computer Gets It Wrong in Medical Admission Test

January 30, 2007


Computer Gets It Wrong in Medical Admission Test


By KAREN W. ARENSON


When Daniel Sonshine, a senior at Brown University, took the Medical College Admission Test on Saturday, he was asked to read a passage on robotic fish in the verbal reasoning part of the exam. Then he was presented with a series of questions about songbirds.


This was not a trick question; it was an error.


“I was completely distraught,” Mr. Sonshine said yesterday. “I was struggling to stay focused, but I was not focusing.”


He was probably not alone. About 800 students who took the exam, known as the MCAT, encountered the mistake.


Another kind of problem surfaced in the College Board’s SAT exams, also given this past weekend. At least one student in South Korea had a part


of the test before it was taken by 326,000 other students, according to the Educational Testing Service, which handles security for the exam.


Raymond Nicosia, privacy protection officer for the testing service, said it was working with investigators in South Korea to find out “what the student had and who else had it.”


Mr. Nicosia said scores for students who had prior information about the test would be canceled. At this point, he added, “all the information we have is that it is a localized situation to Seoul, South Korea.”


Robert F. Jones, a senior vice president at the Association of American Medical Colleges, which oversees the MCAT, generally viewed as the most stressful of the admissions exams, said the error on that test was “something we regret.” “No more than 800” test-takers of about 2,500 were affected, he said.


About 50,000 people take the test each year, some more than once. Mr. Jones added that the error appeared to be “a test publishing problem,” but that he did not yet know how it had come about.


Last weekend was the first time the MCAT was administered only by computer rather than by paper and pencil. The medical college association announced in 2005 that it would move to an entirely computer-based format beginning in 2007, and that it would work with Thomson Prometric to administer the test at its computer testing centers.


Jodi Katz, a spokeswoman for Thomson Prometric, declined to comment on the error.


Mr. Jones, of the association, said he expected that students with the bad test would get their results within 30 days, like other students, because their scores could be extrapolated from the rest of their responses.


“It will not affect their scores,” he said.


But Mr. Sonshine said he thought the painful experience could affect how he had done, and his admission to medical school.


“We are going to get screwed a little bit,” said Mr. Sonshine, who was taking the test a second time because he was not satisfied with the score he received on one section of the exam when he took it last summer.


He said he spent three weeks of his Christmas vacation studying for the exam, from 9 a.m. to 5 p.m. daily, and took five computerized practice tests. When he got to the exam on Saturday, in New Jersey, he whizzed through the first section, on the physical sciences, he said. But his confidence evaporated as he began the second section, verbal reasoning.


Using a technique he had learned from a test coaching company, he looked for a passage to read that seemed easy, to gain momentum. Unfortunately, the one he chose was the one on robotic fish.


“I read through and took some notes,” he said. “Then I went to the questions. ‘The male warbler cries for the female warbler when…?’ I’m starting to freak out.”


When he saw that the next question was also about warblers, he said, he started scrolling frantically through the other sections to see if they were misaligned, too. He then started trying to answer what he could. “Seven minutes were already gone,” he said. “Every fiber of my being is telling me to go outside and say something. At the same time, the clock is ticking, and I haven’t answered a question yet. I struggled through the rest.”


Mr. Jones said that after some students raised questions during the exam, all the Thomson Prometric testing centers were notified and were advised to tell students to ignore the problem section and not to worry. But Mr. Sonshine said he had not been told about the problem.


Some critics of standardized testing said yesterday that while there were advantages to computerized testing, they were not surprised to hear of problems with the MCAT.


“Every time a test has been computerized, there have been huge glitches,” said Robert A. Schaeffer, public education director for FairTest, an advocacy organization that opposes widespread standardized testing.


Mr. Schaeffer said the SAT problem grew out of the College Board’s practice of reusing its tests.


John S. Katzman, chief executive and founder of the Princeton Review, a test coaching company, said the problems with the SAT last weekend were a reminder that security and operational problems “happen even with paper and pencil.”


Mr. Jones said pilot tests last year, including one in August in which about 3,000 students took the computerized version of the MCAT, “went very smoothly.”



It is not unusual that computer glitches were discovered…that is really the norm for any new computer method is used. It’s just ashame that some students will be going through the stress of this learning process (computer learning process, that is). Since I’ve a year and half before I even attempt it, I’m hoping that it will be ironed out by then.


For those OPMs that are planning on taking the MCAT this spring, if you can wait until April or after, it may be better for you.


Just my humble opinion!


Kris

The other lesson is that if something is wrong, speak up right away! Don’t just stew over it! If a section is boogered up, call it to the attention of a proctor and then work on other sections while someone figures out the situation. You’re in control, even when things are beyond your control.

I think the best strategy in that situation would have been to skip this passage and work all the proper passages, and then go back to this one at the end and pick answers that seem closest to correct. There is actually a strategy for doing this–Exam Krackers demonstrated it at their presentation at my school, though of course the sample questions they used lent themselves to this approach.


I feel sorry for the Brown undergrad who allowed them to print his real name in this nationally read newspaper. Using language like “screwed”, he comes off as a bit petulant. OK, sure, he had an unfortunate experience, but it was an error; it’s not like AAMC was out to get him personally.


If I were interviewed by the New York Times, knowing that possibly every admissions person in the country might be interested in this article, I’d want to say something like “It was obviously an error, but I made the most of it by working around it. I am confident that my many hours of preparation have paid off nonetheless and I will receive a competitive score. Medicine is about adapting to unexpected situations, after all.” Talk about turning sh*t into gold!

  • ttraub Said:
I think the best strategy in that situation would have been to skip this passage and work all the proper passages, and then go back to this one at the end and pick answers that seem closest to correct. There is actually a strategy for doing this--Exam Krackers demonstrated it at their presentation at my school, though of course the sample questions they used lent themselves to this approach.

I feel sorry for the Brown undergrad who allowed them to print his real name in this nationally read newspaper. Using language like "screwed", he comes off as a bit petulant. OK, sure, he had an unfortunate experience, but it was an error; it's not like AAMC was out to get him personally.

If I were interviewed by the New York Times, knowing that possibly every admissions person in the country might be interested in this article, I'd want to say something like "It was obviously an error, but I made the most of it by working around it. I am confident that my many hours of preparation have paid off nonetheless and I will receive a competitive score. Medicine is about adapting to unexpected situations, after all." Talk about turning sh*t into gold!



Yeah I think he "screwed the pooch" on that one. A chance to have a PS plastered all over the place and the guy turns it into a pity party. So now most adcoms would look at that and think, if we ever have problems with him on a test we know exactly how he'll react. Screwed himself out of a great oppurtunity. They gave him lemons and he squirts them in his eyes and wah wah wah.....

On the other hand, I'm sure glad I'm waiting till next year to take mine. I figured this would happen. Heck who knows the mistake could actually help them.

Didn’t he say that he did really well in the PS section? What if when he gets his score, he did not do so hot. How will he explain this to the medical schools?


Ah, youth.

When you take the GMAT electronically, you get an estimate right away of your score. It hit mine right on the money. Does the new electronic MCAT do the same?