Robert C. Bowman, M.D.
Primary Source: Assessing Readiness for Medical Education Evolution of the MCAT by William C. McGaghie, PhD JAMA September 4, 2002 Vol 288 No 9
McGaghie noted that the MCAT was developed in 1928 to improve attrition rates which ranged from 5 - 50%. The Medical Aptitude test "combined with national currents toward more rigorous higher educational standards, stricter medical school accreditation policies, and strict student admission procedures" reduced attrition to 7% by 1946.
Prior to reading this article, I gave Flexner far too much of the credit regarding improvements in medical education. Flexner’s Impact on American Medicine Now I give the early MCAT test designers much more credit. According to Flexner, one of the key considerations in the quality of medical education was and is, the quality of the applicants. The earliest MCAT versions did much to stimulate education, college, and the preprofessional system that we have. As we have discussed, there have been both major benefits and major inequities resulting from MCAT use and abuse. Concerns center on who is "different" enough to do poorly but is still qualified to become a physician. This was a theme throughout the article.
The MCAT was not without its critics, even at the earliest stages. McGaghie quotes Thorndike (coined social intelligence now known as emotional intelligence in 1920) as of 1936 as noting that the early MCAT was "better to predict success in the first two years of medical school than success later and throughout life." AAMC's primary goal for the MCAT indeed remains prediction of USMLE 1 passage. Becoming a physician involves more than multiple choice tests as these authors have pointed out.
See Standardized Tests Income Ethnicity
For an excellent discussion of these topics plus gender and education and college impacts, go to Women, Minorities, and Persons with Disabilities in Science and Engineering at http://www.nsf.gov/sbe/srs/nsf99338/frames.htm especially chapter 2 and 3
The non-cognitive approach to admissions has faced great challenges. According to McGaghie, the MCAT contained a section regarding "Understanding modern society" from 1946 - 1962 in an attempt to broaden the preparation. He quotes George Miller in 1961: "knowledge of modern society as measured by the MCAT is not considered to be of major importance in the evaluation of the applicant's intellect…whereas the science score is given great emphasis….On the one hand medical school catalogues and forceful spokesmen…exhort the student to gain breadth of vision, a sociological and humanistic orientation, a liberal education. On the other hand, admissions committees appear for the most part to emphasize academic, particularly scientific achievement." The modern society subset became a general information section in the MCAT version during 1962 - 1977.
The MCAT of 1977 - 1991 eliminated the broad approach and focused more on sciences and cognitive skills. McGaghie noted that the current MCAT "does not contain independent measures of either liberal arts achievement or numeracy although quantitative skills are needed to solve some of the problems…" The current version post 1991 added the writing sample.
MCAT designers in the past have encouraged admissions committees to consider multiple options for the use of the MCAT in conjunction with grades and other evaluation. Efforts to go more broadly regarding testing and evaluation have been a constant theme of spokespersons, but have not been implemented in reality. The Non-Cognitive Working Group of 1976 was headed by Jack Colwill who was the long term chair of family medicine at U of Missouri. He has served the nation in several areas includine the Council on Graduate Medical Education and their workforce studies.As noted in the article: "the Non-Cognitive Working Group died due to a lack of funding combined with skepticism about the utility and reliability of behavioral measurements of personal qualities (from Colwill)."
Comment: Even though the recent mandate from Jordan Cohen involves the non-cognitive area of communications skills, the prevailing attitude is one of skepticism toward the potential benefits and actual usage of this and any other non-cognitive measurements. see Dr. Cohen's comments and my links at MCAT Correlations
McGaghie closed by noting that "the MCAT embodies an enduring set of professional values that have been expressed differently throughout its existence yet recur and endure. The definition of readiness for medical education clearly has an academic component that the MCAT has captured well in different ways in the successive versions described in this report. But it also has professional and personal components, as yet unmeasured or measured poorly via interviews, that scholars have repeatedly have argued are essential to medical education, clinical care, and medical professionalism. The challenge to authors of future MCATs is to rigorously incorporate these nonacademic characteristics into test design and development. The challenge to users of future MCAT data is to take nonacademic characteristics seriously during test score interpretation and use."
End of article summary
Now that the attrition rate of medical students is nearly zero, why aren't we raising serious concerns. Have we gone too far? We know that we need to "take a chance" on some students because of the potential they have to serve the nation in a variety of ways. There are grave and growing concerns regarding the non-cognitive abilities of physicians. Is it possible to entertain a debate regarding the side effects of choosing too intellectual and too cognitive? Can the brilliant be relevant enough and are their enough of this individuals with multiple gifts? Can admissions committees see past their own institutional needs to the needs of society and medicine?
I always get nervous when someone has a zero failure rate. Public health principles have taught me that the last few percent are always the most costly and difficult to obtain. As a physician I know that medical lab tests that identify the highest percentage often are not specific as to what is causing the problem. This leads to more tests. Without incredibly sensitive and specific tests, the cost-benefit ratio is a real problem, beyond just costs to morbidity and mortality from further evaluation. What are the true costs of identifying too many at risk students? Studies note that 90% of those at risk of failure can be identified, but 80% will graduate (Alabama studies). MCAT of 8 - 9, and GPA 3.4 - 3.6 is a grey zone, where MCAT and GPA have much less to contribute.
One of the most disturbing thoughts that I have regarding admissions involves the exclusion of those who could make great contributions to medicine, because of who they are, their experiences, their goals, their motivation, etc. I fear that many are not even attempting a career where they could make great contributions, partly because of the emphasis upon early education, high school, and college. For far too many, the opportunity and impetus for education and career development comes later. They think that they have missed the boat permanently for medicine and other professions.
Medical school admissions are based on the MCAT, grades, the application, and the interview. Studies have highlighted the problems regarding the less objective and non-cognitive areas. In the absence of better measures, better training of committee members, better understanding of the impact of admissions beyond USMLE 1, better information from those who know candidates better or best, and more time spent in admissions deliberations; there will be increasing reliance on MCAT, GPA, and college information. Societal, legal, and medical education pressures assure us that we will continue on this path without some major interventions. Those students who do not do as well on speeded tests, those who do not have the highest grades for a variety of reasons, and those who cannot afford colleges with the best reputations and track records will continue to have less opportunity to become physicians. Family Physicians Are Different
At stake are not just rural doctors, family medicine doctors, and doctors who will serve underserved areas and populations. Another regression that I did involving US private medical schools noted fewer office-based physicians graduating from medical schools with higher MCAT. Just did a regression involving medical schools that graduate the most researchers. Higher MCAT alone explained 55% of the variance. MCAT plus NIH dollars explained 62%. The three are inseparable with all having 0.7 correlations with each other. There are also indications that our admissions process is changing workforce regarding general surgery and other disciplines and possibly the supply of medical educators (decrease in rural and instate admits) . These studies need to go beyond the medical school level to the individual student level, to confirm these indicators. MCAT Correlations
It is time to reconsider admissions decisions. The great successes of the MCAT in previous years are threatened by restrictive use today. However to address this we will need to adopt a different attitude in two areas: our persistent over-association of intellect with status and possible performance our attitude toward medical school attrition.
It is clear that we must risk attrition to gain more physicians that can have a broader impact upon medicine and society. In order to address this area, we must tackle the slavery of medical education debt. Admissions committees greatly fear admitting a student who then cannot graduate, perhaps too much so. Right now things are far too cozy. Why question the direction of medical education? Students are easy to teach if not outright teaching themselves? Why change the tuition-debt-loan triad? Tuition continues to rise and interest rates are low.and medical careers look secure. Changes in these equations will eventually rock the boat. Studies are needed in a number of areas. Also health planners haven't got a clue regarding the long term cost and other consequences of abusing students with incredible debt at their earliest medical exposures. I believe if we had at least some of this information, we would have changed education and medical education long ago.
Robert C. Bowman, M.D. [email protected]
See also Admissions Tracks and proposals