Validity of the MCAT

If any of you have ever wondered if they have ever studied whether or not the MCAT is needed, the answer is yes.
http://www.academicmedicine.org/cgi/content/full/80/10/910

Quote:

If any of you have ever wondered if they have ever studied whether or not the MCAT is needed, the answer is yes.
http://www.academicmedicine.org/cgi/content/full/80/10/910


So the director of the MCAT and employee of the organization that administers the test, thinks the MCAT has predicive value in relationship to USMLE scores? WOW!! I’m COMPLETELY shocked!
As a former epidemiologist I KNOW stats can be skewed to represent whatever argument you’re trying to make. And because I know so many low/mid 20’s MCAT folks that went on to rock their boards, I just don’t buy that there’s a strong coorelation.

Hi there,
The MCAT is a test of problem solving, something that you will do every day of your life as a physician. It has relative predictive value in terms of USMLE but nothing absolute. It is one of the ultimate standardized tests and something of a measure of standardized test-taking but again, not an absolute indicator of future performance in medical school or as a physician.
The one thing that the MCAT does provide is an indicator of how a particular applicant measures against other applicants who took this test. Undergraduate institutions vary greatly but the MCAT is something of a constant. It is but one more tool with which to evaluate applicants.
Many of my admissions committee colleagues spend at least the first meetings discussing how they believe that the MCAT is meaningless in today’s atmosphere. I have found that it is valid only in the sense of how well an applicant can apply knowledge from pre-med coursework to problem solving. I do find some validity to this test but it is not the major criterion that I use to judge an applicant just as I do not use USMLE as the sole criterion to judge a residency applicant.
No school can afford to admit an applicant that cannot get through the curriculum. Many schools that were willing to accept lower MCAT scores and GPAs were finding that the failure rate of matriculants was getting too high and thus more emphasis has been given to those “numbers” of late.
There are some stories of matriculant with low MCAT and marginal GPA excelling in medical school but in general, the applicants that we have accepted with lower GPAs and MCATs have struggled with the curriculum.
There is no absolute predictor of medical school success. Most of the people who matriculate in medical school have been pretty high achievers. The medical school curriculum be it problem-based learning, systems-based or classical is a huge adjustment for anyone.
Until something better comes along, the MCAT is with us and getting more challenging as more students take prep courses and spend months preparing for this exam. GPA has about the same predictive value and MCAT + GPA is somewhat indicative of ability to master curriculum but not absolute.
Natalie

Hey
If there was a real relation to entrance exams like the MCAT and quality of physician then every country outside of the USA would have horrible physicians or at least inferior. This isnt the case. There are even some schools in the USA and Canada which dont use the MCAT
While i agree it is a test of problem solving, personally, i dont beleive it is realistic in regards to physician but only for the USMLE’s. So what does that mean? Well lets see. While i dont believe that a good MCAT (or any MCAT) = good physician as evidenced by other countries which dont use admission testing. Now, lets put that in perspective. If you dont do well on the USMLE’s you cannt be a physician in the USA.
So at the end of the day, as horrible an indicator as the MCAT is or the USMLE’s for that matter, these are best options we have in the USA. That being said the MCATS are the most important indicator for med school because they are required!
clear as mud

Which schools don’t require the MCAT? I think that’s the case for students in an accelerated program through their university?

Quote:

Hey
If there was a real relation to entrance exams like the MCAT and quality of physician then every country outside of the USA would have horrible physicians or at least inferior…
While i dont believe that a good MCAT (or any MCAT) = good physician as evidenced by other countries which dont use admission testing.



But which countries are you talking about, that don’t use admissions testing? Most countries have some form of examination used to evaluate applicants. In a lot of countries, the exams scores are ALL that count. Whether it is a university graduation exam or a “standardized” test, still there are requirements that applicants must fulfill. Did you know that in England, there are at least 3 separate standardized tests that are the equivalent of the MCAT? Different med schools require different ones! Australia has the GAMSAT. To get into med school in Denmark, you have to graduate from high school at a certain rank in your class. So it isn’t like other countries just waive these requirements. The reason most other countries probably don’t have an exact equivalent of the MCAT is that their universities tend to be public and centrally run and therefore, a student’s university performance is a more consistent measure of their academic achievement.
If anything I think the US is unique in considering applicants’ personal qualities, volunteer experiences, and their undergraduate grades. We also have a WAY better opportunity to go back to school as adult premeds. In some countries, this just isn’t possible. And at least with the MCAT, you can retake it if needed!
I’m not trying to make some huge defense for the MCAT, I’m just trying to say that the grass isn’t necessarily greener on the other side…

Actually other countries are MUCH harder as far as entry into medical school! forget about it if you are past 30! msot countries have a very rigorous exam AFTER high school where you must rank in the higher percentiles to EVEN be eligable to apply to medical school. This exam is NO joke and it is not m/c btw. It is essay that encompass anything learned during high school. The high school in Europe is MUCH more difficult than here in the US it compares to undergrad. Again, there is no perfect way but the MCAT is a good equalizer where it does not matter where you went to school but how well you can dissect information. There HAS to be some way to differentiate folks…we cannot just let anyone in “becuase they are nice” and would listen to their patients. Heck I want to play in the NBA but at less than under five feet that is just not possible.

The MAIN problem I have with the MCAT is that not every question is graded. For example, if a certain number of testers get a question wrong, it’s NOT figured into the total score. It also appears that they REALLY curve the scores as well. So if it’s supposed to be a test of your ability to disect info, why the need to throw out certain questions and curve the scores to fit a perfectly symetrical normal curve? It seems to me that based on the academic qualities of people who take the MCAT, it should be skewed in the right side of a normal distribution curve. But then most people would have high scores and the system of Kaplan, EK would dissolve.

The MCAT is not only a test of your basic scientific knowledge. If this were the case, there would only be discrete questions. The MCAT also tests your ability to critically think. To take the information given to you and to use the information you already know and apply it. this is why the verbal reasoning section is so critical and why the science sections are made up mainly of reading passages.

Quote:

The MCAT is not only a test of your basic scientific knowledge. If this were the case, there would only be discrete questions. The MCAT also tests your ability to critically think. To take the information given to you and to use the information you already know and apply it. this is why the verbal reasoning section is so critical and why the science sections are made up mainly of reading passages.


Some of you may not be old enough to remember but most of (if not all) of the MCAT used to be discrete questions except the verbal section, of course. Personally, I think the MCAT is a test about how you take the MCAT, far more than what you know. You can “know” a lot but if doesn’t happen to be on your MCAT test form, then you’re in trouble.
Just curious, are the USMLE’s curved as well? Does it like the MCAT tend, to vary in what subject matter is emphasize depending on the form you receive or the time you take it?

Hey Path., if you read the paper, you probably saw what I saw, which is a hoot. First of all, this was not an experiment done on an appropriate population. The students who got into the study had already gotten into medical school. How did they do that? The MCAT maybe? So all this may be showing is a slight difference between people who did well on the MCAT and people who did REALLY well on the MCAT, and it doesn’t begin to show the predictive capability of any test unless a large population of low-scorers are also let into med school to see how they will do. Did anybody look at the data? They say that for predicting cumulative GPA in med school, MCAT scores can explain away 35% of the score variability. Wonder what explains the other 65%? Ya, me too. I think I have that, do you have it? Who knows? When they use MCAT and uGPA together, they can only explain only a whopping 50%!! I noticed that in all of these studies, u-school “selectivity” didn’t amount to a hill of beans. But then again, they only used average SAT scores. What about the people who went to ACT schools, or (gasp!!) the dreaded community college? I am assuming that they were taken out of the entire study, since they didn’t have an SAT score to utilize. Remember this the next time some admissions counselor points out that you went to Joe Schmoe CC and show them how it doesn’t matter (or more correctly, that they have no meaningful data here). The study gets better. There is a big difference between med schools. On the average, MCAT scores and uGPA account for 50%, but the school values range from 23% to 66%. In other words, for some schools, over 75% of the variability in their GPA is due to something else other than the scores they all use to let us in. The author(s?) also are vague when it comes to predictive capability of the 3-year GPA, which ranges 18-42% for the MCAT and 29% when the MCAT and the uGPA both are used. They should have reported stuff consistently. The data showing correlation to USMLE is a little better, once again showing if you can take one non-predictive test, you can take them all. (Actually, I don’t really know if the USMLE is correlated with anything.) MCAT scores account for only 41% of the variability, and MCAT and GPA together account for only 41% of the variability. Says alot for GPA doesn’t it?

Megboo,
I attend USF in Florida, and like many schools we have a 7 year Honors Med program. Admission to med school is “guaranteed”, but only “if” you maintain a high GPA as well as do well on the MCAT. I would be curious too about which schools do not require the MCAT in the US. The only schools I know of are overseas/islands etc. As for the 7 year med program, great idea, but only for those entering college for the first time since high school…kinda omits older pre-meds. Another door closed…okay, just another wall to figure out how to get around!!

Quote:

Hey Path., if you read the paper, you probably saw what I saw, which is a hoot. Says alot for GPA doesn’t it?


I just wanted to add that it makes sense that if a person’s GPA AND MCAT are below those of average applicants, that they may struggle with the curriculium. But does anyone really know anyone who got into med school with weakness in BOTH of these key areas?

Actually, according to the study, MCAT alone accounts for 41% of the variability in the USMLE score, and MCAT and GPA together, is also 41%, which means that the GPA counts for essentially nothing. I’d have to go back and reread, since I am not sure if in this section they were talking about uGPA or med school GPA. It could be, that for the students who were in this study, being preselected by the fact that they were admitted into med school, had a greater-than-average MCAT score and a greater-than-average GPA. If fact, they could be correlated enough so that if they ran a stepwise regression starting with GPA, and then adding MCAT variable, the MCAT score would show up as fairly insignificant. Of course, they wouldn’t do that, since the point of the study is to show that MCAT is worthwhile. Of course, this type of data has problems. This is why people do designed experiments.